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PREFACE 


This  report  documents  the  first  Federal  Aviation  Administration 
controller  evaluation  of  a  group  of  four  terminal  air  traffic 
control  services  and  functions  which  have  been  designed  for 
implementation  on  the  Data  Link  air-ground  communication  system. 
The  research  was  conducted  at  the  Federal  Aviation  Administration 
(FAA)  Technical  Center  by  the  Airborne  Collision  Avoidance  and  Data 
Link  Systems  Branch,  ACD-320. 
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EXECUTIVE  SUMMARY 


INTRODUCTION. 

The  Federal  Aviation  Administration  (FAA)  is  pursuing  an  initiative 
to  develop  and  implement  a  Data  Link  system  intended  to  enhance 
communications  between  ground-based  air  traffic  control  (ATC)  and 
aircraft  operating  within  the  National  Airspace  System  (NAS) .  By 
providing  digital  information  transfer  along  with  the  ability  to 
discretely  address  individual  receivers,  Data  Link  is  expected  to 
relieve  frequency  congestion  on  existing  voice  radio  channels  while 
increasing  the  safety  and  productivity  of  ATC  operations. 

In  order  to  insure  that  the  introduction  of  Data  Link  will  have  an 
optimally  positive  impact  on  ATC,  the  FAA  Technical  Center  is 
conducting  a  program  of  research  and  development  to  guide  the 
design  of  effective  ATC  services,  evaluate  their  impact  on  system 
performance,  and  promote  the  integration  of  Data  Link  with  the  air 
traffic  controllers  who  will  use  it.  Prior  efforts  under  this 
program  have  focused  on  the  design  and  operational  evaluation  of 
an  initial  group  of  ATC  services  for  the  en  route  ATC  environment 
(Talotta,  et  al.,  1988,  1989,  1990).  This  report  presents  the 
findings  of  the  first  FAA  Technical  Center  investigation  of 
terminal  ATC  services  developed  for  transmission  using  Data  Link 
technology . 

The  following  ATC  services  and  functions  services  were  defined  and 
conceptual ly  designed  by  the  terminal  control] er  subgroup  of  the 
Air  Traffic  Data  Link  Validation  Team  ( ATDLVT) :  Transfer  of 
Communication,  Initial  Contact,  Terminal  Information,  and  Menu 
Text.  For  the  present  study,  these  initial  capabilities  were 
implemented  in  the  Data  Link  Test  Bed  using  the  currently 
operational  Automated  Radar  Terminal  System  (ARTS)  IIIA  equipment. 

OBJECTIVES. 

The  overall  goal  of  this  research  was  to  establish  a  developmental 
baseline  for  terminal  services  in  the  Data  Link  Test  Bed  which  will 
support  design  evolution,  operational  evaluation,  and  eventual 
field  implementation.  Specifically,  the  study  was  conducted  to 
determine  the  acceptability  of  the  preliminary  service  designs,  to 
identify  requirements  for  design  modifications  and  enhancements, 
and  to  elicit  controller  estimates  of  the  operational  suitability, 
user  acceptance,  and  workload  impact  of  these  services  in  the 
terminal  ATC  environment. 

APPROACH. 

The  objectives  of  the  study  were  addressed  in  a  series  of  training 
and  ATC  simulation  test  sessions  conducted  at  the  ARTS  IIIA 
workstations  in  the  Data  Link  Test  Bed.  During  nesting,  nine 
terminal  air  traffic  control  specialists  participated  in  a 
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simulation  scenario  involving  arrivals  and  final  approaches.  The 
subjects  completed  a  detailed  design  review  of  each  of  the  four 
services  and  compared  the  current  voice-only  system  to  a  system 
supplemented  by  the  initial  Data  Link  capability. 

PRIMARY  RESULTS . 

The  controllers  who  participated  in  this  study  indicated  that  Data 
Link  would  have  strongly  positive  effects  on  terminal  operations 
if  implemented  in  the  present  ATC  system.  These  effects  were 
estimated  to  include  a  significant  reduction  in  voice  radio 
frequency  congestion,  simplified  transfer  of  lengthy  advisory 
messages  and  clearances,  and  a  reduction  in  communication  errors. 
Furthermore,  quantified  projections  of  average  controller  workload 
in  a  60  percent  Data  Link  aircraft  equipage  environment  were  found 
to  be  significantly  lower  than  comparative  estimates  of  workload 
in  the  exclusively  voice  radio  communications  environment. 

The  four  services,  as  modified  by  controller  inputs  obtained  during 
the  study,  received  high  ratings  on  the  dimensions  of  operational 
effectiveness  and  estimated  accepatabi 1 ity  to  field  controllers. 
Enhancements  and  design  changes  agreed  upon  by  the  subjects 
included  the  addition  of  transaction  status  and  history  lists,  as 
well  as  increased  flexibility  in  the  design  and  accessibility  of 
items  contained  in  the  Menu  Text  and  Terminal  Information  Service 
lists. 

Finally,  the  findings  of  the  study  indicated  that  continued 
research  attention  will  be  required  in  the  area  of  controller  human 
factors  issues  to  guarantee  the  effectiveness  of  Data  Link  services 
in  a  broa^  range  of  operational  terminal  ATC  conditions.  These 
issues  include  the  control  of  visual  search  demands  in  the  use  of 
menu  functions,  insurance  of  optimal  information  content  in 
transaction  status  displays,  and  minimization  of  short  term  memory 
requirements . 

RECOMMENDATIONS . 

The  results  of  this  study  demonstrated  that  the  initial  terminal 
services  have  the  potential  to  reduce  frequency  congestion  and 
controller  workload,  and  are  compatible  with  the  existing  NAS 
terminal  workstation  and  computer.  Therefore,  it  is  recommended 
that  continued  developmental  and  operational  evaluation  research 
be  pursued  in  the  Data  Link  Test  Bed  to  support  the  implementation 
of  these  services  at  the  earliest  possible  date.  As  a  part  of  this 
effort,  work  should  be  initiated  to  verify  the  technical 
feasibility  of  implementing  Data  Link  in  the  present  and  future 
operational  ATC  systems,  including  the  conduct  of  empirical  tests 
of  ARTS  1 1 1 A  capabilities  as  outlined  in  appendix  A  of  this  report. 
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1. 


INTRODUCTION. 


1.1  PURPOSE. 

This  document  presents  the  results  of  the  first  Federal  Aviation 
Administration  ( FAA)  Technical  Center  investigation  of  terminal  air 
traffic  control  (ATC)  services  developed  for  transmission  using 
Data  Link  technology.  Preliminary  designs  for  four  ATC  services 
were  implemented  in  the  currently  operational  National  Airspace 
System  (NAS)  Automated  Radar  Terminal  System  (ARTS)  IIIA  computer 
and  ATC  workstation  for  review  and  evaluation  by  air  traffic 
controllers.  The  controllers  participated  in  simulated  terminal 
airspace  test  trials  to  assess  the  utility  of  the  Data  Link 
services,  recommend  requirements  for  service  design  changes,  and 
provide  initial  views  regarding  the  potential  impact  of  Data  Link 
on  terminal  operations  in  the  NAS.  It  is  expected  that  this  study 
will  be  the  first  in  an  iterative  series  of  design  simulation  tests 
which  will  culminate  in  a  full-scale  operational  evaluation  and  the 
production  of  design  specifications  for  an  operational  Data  Link 
communications  system  for  the  terminal  ATC  environment. 

1.2  BACKGROUND. 

1.2.1  The  Air-Ground  Communication  Problem. 

In  response  to  the  phenomenal  growth  of  air  traffic  in  the  United 
States,  the  FAA  has  begun  to  develop  and  implement  a  broad  range 
of  initiatives  aimed  at  updating  and  enhancing  ATC  technology.  Many 
of  these  efforts  are  focused  on  improving  the  quality  and  quantity 
of  information  that  will  be  needed  to  increase  safety  and 
productivity,  and  on  insuring  that  this  information  is  reliably  and 
accurately  transferred  among  the  computers  and  humans  that  form  the 
major  components  of  the  ATC  system. 

One  of  the  primary  information  transfer  problems  that  constrains 
the  capacity  of  the  current  ATC  system  is  the  inherently  limited 
communication  channel  that  exists  between  the  air  traffic 
controller  and  the  aircraft  pilot.  Because  this  voice  radio  link 
operates  in  a  broadcast  mode  between  a  single  controller  and  all 
aircraft  operating  in  the  airspace  under  his  control,  frequency 
congestion  is  a  common  occurrence  when  the  volume  of  air  traffic 
increases.  Such  saturation  of  the  communications  channel  affects 
the  performance  of  the  ATC  system  by  preventing  the  timely  issuance 
of  clearances  and  by  restricting  the  vital  exchange  of  information 
upon  which  safe  and  efficient  operation  of  the  NAS  depend. 

In  addition  to  the  limitations  that  it  imposes  through  frequency 
congestion,  the  voice  radio  channel  has  been  identified  as  a  major 
contributor  to  errors  in  the  ATC  system.  The  FAA  has  noted  that 
as  many  as  23  percent  of  all  operational  errors  are  caused  either 
directly  or  indirectly  by  communications  mistakes  (New  York  Times, 
1988).  Similarly,  compilations  of  voluntary  reports  provided  to 
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the  Aviation  Safety  Reporting  System  by  pilots  and  controllers  have 
shown  that  a  majority  of  all  potentially  hazardous  incidents  that 
are  filed  implicate  ineffective  verbal  information  transfer 
(Billings  and  Reynard,  1981). 

Investigations  of  the  nature  of  prevalent  communications  errors 
demonstrate  that  they  are  typically  the  result  of  an  interaction 
between  the  characteristics  of  the  voice  radio  system  and  the 
inherent  perceptual  and  cognitive  characteristics  of  its  human  user 
( Shingledecker ,  1990).  Acoustic  confusions,  alphanumeric 
transpositions,  misinterpretation  due  to  pronunciation  and 
phraseology  problems,  poor  memory  for  transient  speech 
presentations  of  ATC  information,  and  blocking  of  the  radio  channel 
caused  by  improper  keying  techniques  are  common  sources  of  human- 
induced  error  found  by  these  studies.  In  addition,  many  errors 
seem  to  be  potentiated  by  the  frequency  congestion  problem  as  users 
experience  difficulty  in  monitoring  for  relevant  messages  on  the 
crowded  radio  channel,  and  become  reluctant  to  clarify  suspected 
confusions  in  order  to  avoid  further  congestion. 

1.2.2  Data  Link  Solutions . 

Data  Link  is  a  digital  communications  technology  which  is  being 
developed  as  a  supplement  to  traditional  voice  radio  for  ATC 
communications.  As  shown  in  figure  1,  Data  Link  communications  can 
be  supported  by  several  transmission  media.  These  include  very 
high  frequency  ( VHF)  radio,  satellite  links,  and  the  Mode  Select 
(Mode  S)  secondary  surveillance  radar  system  currently  proposed  by 
the  FAA  for  ATC  Data  Link  communications.  These  multiple  links 
will  be  integrated  within  a  common  Aeronautical  Telecommunications 
Network  to  provide  seamless  air-ground  communications  throughout 
the  NAS. 

Regardless  of  the  specific  method  used  to  create  the  channel,  Data 
Link  communications  are  <:* ist inguished  from  traditional  voice  radio 
links  in  two  essential  ways.  First,  unlike  analogue  voice 
messages,  Data  Link  messages  consist  of  digitally  coded 
information.  Thus,  data  may  be  entered  for  transmission  either 
manually,  or  by  direct  access  to  information  contained  in  airborne 
or  ground-based  computers.  Furthermore,  the  capability  of  a 
digital  system  to  provide  ;■ automatic  error  checking  of  sent  and 
received  messages  makes  Data  Link  a  highly  reliable  system  which 
is  not  as  susceptible  to  degradation  by  interfering  noise  sources 
compared  to  the  existing  voice  communication  frequencies. 

The  second  way  in  which  Data  Link  differs  from  the  voice  radio 
channel  is  its  capability  to  discretely  address  individual 
receivers.  Unlike  the  simplex  radio  system  which  permits  only  a 
single  speaker  to  transmit  on  the  broadcast  frequency  at  any  point 
in  time,  Data  Link  messages  can  be  sent  selectively,  and 
transmission  rates  are  not  artificially  bounded  by  the  speaking  and 
listening  rates  of  the  user.  As  a  result,  Data  Link  channels  can 
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FIGURE  I.  THE  DATA  LINK  SYSTEM 
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have  a  much  higher  capacity  than  voice  channels  and  critical 
messages  sent  by  a  controller  are  assured  of  receipt  only  by  the 
intended  aircraft. 

These  features  of  Data  Link  offer  significant  promise  for 
alleviating  both  frequency  congestion  and  errors  that  currently 
impair  air-ground  ATC  communications.  Demands  on  the  voice  channel 
should  be  relieved  in  proportion  to  the  number  of  weather  and  ATC 
services  that  are  assigned  to  the  Data  Link  system.  In  addition, 
by  automating  o  simplifying  pilot  and  controller  functions  in  the 
communication  pj ocess  that  are  subject  to  error,  Data  Link  should 
improve  the  overall  effectiveness  of  information  transfer.  For 
example,  using  Data  Link  it  will  be  possible  to  reduce  ambiguous 
message  transmissions  by  storing  standard  clearances  in  computer 
memory  for  simplified  uplink  to  an  aircraft;  failures  to  detect 
messages  and  accidental  acceptance  of  clearances  by  unintended 
aircraft  will  be  eliminated  by  discrete  addressing?  interpretation 
errors  should  be  reduced  by  the  availability  of  a  persistent  and 
recallable  visual  display  of  the  received  data;  and  the  system  will 
automatically  verify  the  integrity  of  a  message  without  human 
intervention. 

1.2.3  Data  Link  Design  and  Evaluation  Research. 

As  noted  above,  the  technical  characteristics  of  Data  Link  have  the 
capability  to  significantly  enhance  the  safety  and  productivity  of 
the  ATC  system.  However,  Data  Link  also  will  introduce  a  profound 
change  in  the  way  in  which  ATC  tasks  are  accomplished  by 
controllers,  and  in  the  way  aircrew  will  receive  and  respond  to  ATC 
instructions.  Because  of  this,  the  ultimate  success  of  Data  Link 
will  be  critically  dependent  on  the  extent  to  which  it  is  employed 
to  create  an  effective  communications  system  that  is  fully 
integrated  with  its  human  users  and  with  the  full  range  of  tasks 
that  they  are  required  to  perform. 

Recognition  of  the  need  to  consider  operational  suitability  and 
human  factors  issues  as  primary  drivers  of  the  design  process 
prompted  t\e  FAA  Technical  Center  to  initiate  a  program  of  manned 
simulation  research  to  guide  the  development  of  Data  Link  ATC 
services.  The  overall  goals  of  this  research  are  to  (1)  define 
useful  Data  Link  services,  (2)  determine  the  user  information 
requirements  for  Data  Link  communications,  (3)  develop  display 
formats,  data  entry  methods  and  procedures  which  promote  efficient 
controller  performance,  and  (4)  evaluate  the  impact  of  Data  Link 
services  on  both  human  and  system  performance. 

The  Data  Link  Test  Bed  was  assembled  at  the  FAA  Technical  Center 
to  address  these  goals.  The  Test  Bed  is  a  laboratory  facility 
which  uses  actual  NAS  equipment  in  conjunction  with  simulation 
computers  to  create  a  system  capable  of  realistically  exercising 
Data  Link  applications  in  an  end-to-end  fashion.  In  its  current 
form,  the  Test  Bed  is  composed  of  the  NAS  en  route  and  terminal 
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laboratories,  the  NAS  System  Simulation  Facility  (NSSF) ,  and  the 
Data  Link  laboratory  (figure  2).  The  NAS  laboratory  includes  the 
HOST  computer  system  used  for  en  route  ATC  data  processing  as  well 
as  its  primary  terminal  counterpart,  the  ARTS  IIIA  system.  Both 
computers  are  linked  to  several  suites  of  their  respective 
operational  controller  workstations  which  are  used  to  display  radar 
data  and  to  enter  system  inputs. 

The  NAS  laboratory  is  linked  to  the  NSSF  through  the  ATC  computers. 
The  NSSF  permits  the  NAS  laboratory  systems  to  act  as  functioning 
control  facilities  by  providing  simulated  radar  data  and  voice 
radio  inputs  from  simulation  pilots  operating  from  computer 
terminals.  Alternatively,  the  ARTS  and  HOST  portions  of  the  NAS 
laboratory  can  be  used  as  self-contained  simulation  systems  using 
the  training  functions  included  within  the  operational  systems. 
In  this  configuration,  pilot  functions  are  performed  by  simulation 
operators  working  at  additional  controller  workstations. 

The  Data  Link  laboratory  -houses  a  VAX  11/750  computer  which  acts 
as  an  emulation  of  the  future  ground  Data  Link  processor.  The  VAX 
computer  supports  digital  communication  between  simulation  pilots 
and  controllers.  It  can  also  provide  two-way  communication  between 
controllers  and  high-fidelity  aircraft  simulators  or  actual 
airborne  systems  using  Mode  S  or  any  other  installed  Data  Link 
technology. 

The  central  thrust  of  Data  Link  research  in  the  test  bed  is  manned 
simulation  research  aimed  at  defining  and  testing  designs  for  ATC 
services.  The  general  approach  used  in  this  research  employs  a 
two-stage  methodology.  In  the  first  stage,  field  controllers  who 
are  members  of  the  Air  Traffic  Data  Link  Validation  Team  ( ATDLVT) 
participate  in  highly  controlled,  part  task  simulation  studies  to 
reduce  the  number  of  service  design  options  to  a  reasonable  set. 
These  mini  studies  (Talotta,  et  al.,  1988,  1989)  focus  on  formal 
testing  and  review  of  display  and  procedural  options  rather  than 
on  high-fidelity  simulation  of  the  ATC  environment.  Test  scenarios 
range  from  basic  presentations  of  two  or  three  aircraft  to 
simplified  air  traffic  problems  derived  from  actual  controlled 
airspaces.  Using  an  iterative  approach,  feedback  from  the 
controllers  is  used  to  modify  the  designs  for  further  test. 

In  the  second  stage  of  the  process,  operational  evaluation  studies 
are  conducted  to  exercise  refined  service  designs  in  the  context 
of  full-scale  ATC  scenarios.  The  goals  of  these  studies  are  to 
confirm  the  optimality  of  the  finalized  service  configurations  and 
to  examine  controller  performance  issues  which  can  only  be 
addressed  under  test  conditions  approaching  the  realism  of  an 
operational  environment. 
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FIGURE  2.  DATA  LINK  TEST  BED 


1.2.4  En  Route  Development. 


Research  and  development  efforts  in  the  Data  Link  Test  Bed  began 
with  the  en  route  portion  of  the  ATC  system.  Two  mini  studies  were 
conducted  to  refine  the  transfer  of  communication  and  altitude 
assignment  services  as  well  as  a  menu  text  function  for  uplinking 
interim  altitudes  and  an  unformatted  free  text  function  (Talotta, 
et  al.,  1988,  1989).  These  efforts  culminated  in  an  operational 
evaluation  in  which  complex  test  scenarios  and  high  levels  of  air 
traffic  were  used  to  test  the  services  under  20  percent  and  70 
percent  levels  of  Data  Link  aircraft  equipage  (Talotta,  et  al., 
1990) .  The  operational  evaluation  study  clearly  demonstrated  the 
benefit  of  the  initial  en  route  services  on  frequency  congestion 
with  a  reduction  in  controller  voice  transmissions  of  up  to  41 
percent  and  in  channel  occupation  time  of  up  to  45  percent.  This 
result  was  achieved  at  no  observed  loss  in  controller  performance 
or  increase  in  controller  workload  ratings. 

Currently,  en  route  development  is  expanding  to  the  design  and 
testing  of  controller  interfaces  for  speed  and  heading  clearances, 
the  processing  of  pilot  downlinks,  and  procedures  for  the 
utilization  of  assistant  controllers. 

1.3  DATA  LINK  FOR  THE  TERMINAL  ENVIRONMENT. 


1.3.1  Terminal  Data  Link  Issues. 

The  potential  benefits  of  Data  Link  communication  technology  to 
terminal  ATC  operations  can  be  accrued  in  the  en  route  environment. 
At  present,  the  demands  at  busy  airports  often  can  result  in  a 
controller  engaging  in  prolonged  periods  of  non-stop  verbal 
communication  to  convey  all  of  the  clearances  needed  to  guide  the 
pilots  of  arriving,  departing,  and  transient  aircraft.  In 
addition,  the  requirement  to  convey  lengthy  advisory  messages  to 
aircraft  entering  the  terminal  area  rapidly  expends  the  limited 
communication  time  available  to  tactically  control  closely  spaced 
aircraft  on  the  approach  and  departure  flightpaths. 

While  the  need  to  reduce  frequency  congestion  is  similar  in  the 
terminal  and  en  route  environments,  the  problem  of  designing  an 
effective  Data  Link  system  for  the  two  is  quite  different.  In 
general,  terminal  operations  are  more  sensitive  to  timing  issues 
than  en  route  operations.  Because  of  this,  communications 
functions  assigned  to  Data  Link  must  be  carefully  selected, 
designed,  and  tested  to  ensure  that  transmission  delays  or  display 
clutter  do  not  interfere  with  controller  performance  requirements. 
In  addition,  unlike  some  en  route  clearances,  current  terminal 
procedures  do  not  require  the  use  of  keyboard  inputs  to  update  the 
ATC  computer  data  base.  For  this  reason,  the  need  to  develop 
efficient  methods  for  entering  and  uplinking  control  messages  is 
essential . 
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1.3.2  Initial  Terminal  Services. 


Drawing  from  their  experience  as  terminal  controllers  and  an 
awareness  of  the  design  issues  outlined  above,  the  terminal 
subgroup  of  the  ATDLVT  met  with  FAA  engineers  and  supporting 
contractors  in  a  series  of  meetings  held  in  1989  and  1990  to  define 
an  initial  group  of  ATC  services  suitable  for  the  terminal 
environment.  The  following  services  were  designed  during  these 
meetings,  and  preliminary  versions  were  demonstrated  using  a  rapid 
prototyping  system  at  The  MITRE  Corporation: 

a.  Transfer  of  Communication.  Transfer  of  communication  is 
the  message  sent  to  an  aircraft  after  track  control  has  been 
accepted  by  the  new  sector  which  instructs  the  pilot  to  change 
radio  frequencies  in  order  to  communicate  with  the  new  controller. 
Using  the  designed  Data  Link  service,  this  message  is  automatically 
prepared  by  the  ATC  computer  and  uplinked  either  automatically  or 
upon  a  controller  input  action. 

b.  Initial  Contact.  When  an  aircraft  receives  a  new  radio 
frequency,  current  ATC  procedures  require  the  pilot  to  contact  the 
new  controller  and  to  report  the  aircraft's  currently  assigned 
altitude.  With  Data  Link,  after  the  new  controller  obtains  Data 
Link  communication  eligibility,  a  request  for  the  initial  contact 
report  is  sent  automatically  to  the  aircraft,  and  the  downlinked 
report  is  presented  to  the  controller  on  the  radar  display. 

c.  Terminal  Information  Service.  When  arriving  aircraft  enter 
a  terminal  airspace,  they  are  typically  given  a  report  of  the 
terminal  operating  conditions  and  of  the  approach  clearance  that 
they  can  expect  to  receive.  Using  Data  Link,  these  commonly 
lengthy  messages  are  provided  by  a  simple  controller  action  to 
initiate  the  uplink. 

d.  Menu  Text.  Menu  text  is  a  Data  Link  function  which  permits 
the  controller  to  select  and  uplink  a  commonly  used  clearance  from 
a  menu  list  displayed  on  the  radar  screen.  In  the  service  design 
tested  in  this  study,  such  clearances  included  speed,  heading,  and 
altitude  change  instructions. 

1.3.3  Feasibility  of  Implementation  in  ARTS. 

As  discussed  earlier  in  this  introduction,  the  approach  taken  by 
the  FAA  Technical  Center  to  Data  Link  service  development  has 
focused  on  the  use  of  operational  ATC  computers  and  workstations 
to  test  the  human  factors  of  digital  communications  technology  for 
ATC  applications.  The  primary  advantage  of  this  approach  is  that 
it  provides  controllers  with  familiar  equipment  and  a  test 
environment  which  permits  them  to  make  maximum  use  of  their 
operational  experience  to  judge  the  suitability  and  acceptability 
of  the  service  designs  under  evaluation.  Because  of  this,  the 
results  derived  from  the  approach  are  believed  to  be  valuable  not 
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only  for  potential  implementation  of  Data  Link  in  currently 
operational  NAS  equipment,  but  also  for  the  eventual  implementation 
of  Data  Link  in  future  equipment  being  developed  under  the  FAA 
Advanced  Automation  System  (AAS) . 

At  the  present  time,  it  is  uncertain  whether  Data  Link  ATC  services 
will  be  installed  in  ARTS  or  be  delayed  until  the  AAS  terminal 
workstation  and  computer  enhancements  are  implemented.  The 
advanced  equipment  is  being  designed  with  sufficient  computing 
power  and  display  capabilities  to  support  Data  Link  functionality. 
However,  because  of  the  limited  capacities  of  the  aging  ARTS 
system,  consideration  must  be  given  to  the  technical  feasibility 
of  exercising  the  option  to  implement  the  initial  Data  Link 
terminal  services  in  the  near  term. 

Appendix  A  presents  the  results  of  an  analysis  of  the  projected 
impact  on  the  ARTS  IIIA  operational  system  from  the  installation 
of  the  Data  Link  services  which  were  implemented  for  the  initial 
testing  presented  in  this  report.  The  analysis  did  not  include 
projections  of  the  communications  overhead  associated  with  the 
Aeronautical  Telecommunications  Network  (ATN)  implementation  that 
is  being  specified  for  Data  Link  communications.  The  effect  of  the 
communications  requirements  on  processing  and  memory  resources 
depends  upon  the  selected  system  configuration.  Even  if  the  ARTs 
IIIA  would  be  required  to  handle  all  of  the  communications 
processing  as  an  end  state  system,  the  communications  functions 
could  be  accomplished  by  additional  processor  and  memory  resources 
independent  of  the  ARTS  IIIA  processor  and  memory.  With  these 
assumptions,  the  analysis  provides  an  empirical  estimate  of  the 
display,  data  entry,  and  data  processing  requirements  to  provide 
the  initial  Data  Link  services.  The  analysis  indicated  that  the 
services  would  require  no  more  than  10K  of  the  80K  to  128K  of 
unused  memory  available  under  the  most  recent  ARTS  IIIA  upgrade. 
Likewise,  a  five  percent  maximum  increase  in  processing 
requirements  was  predicted,  which  appears  to  be  well  within  the 
capabilities  of  the  current  upgrade.  Although  the  Data  Link 
services  would  place  significant  additional  demands  on  the  ARTS 
display  capabilities,  these  were  also  projected  to  present  no 
problem  when  using  the  latest  Full  Digital  ARTS  Display  (FDAD) 
system. 

In  summary,  the  analysis  indicates  that  the  ARTS  IIIA  with  planned 
upgrades  has  additional  capacity  to  allow  implementation  of 
additional  functions  such  as  Data  Link.  The  expected  processing 
and  memory  utilization  required  to  implement  Data  Link  is  well 
within  the  expected  processor  and  memory  capacities.  The  physical 
and  functional  interconnection  of  the  ARTS  system  to  the  Data  Link 
system  and  the  communications  functions  required  to  accomplish  the 
interface  needs  additional  investigation  to  determine  the  impact 
on  the  ARTS  system.  The  conduct  of  confirmatory  engineering  tests 
would  be  warranted  if  the  option  to  implement  Data  Link  in  the  ARTS 
system  is  exercised. 
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1.3.4  Organization  of  the  Report. 


The  following  sections  of  this  report  present  the  research 
methodology  that  was  used  and  the  findings  that  were  obtained  in 
the  first  FAA  Technical  Center  controller  evaluation  study  of  Data 
Link  terminal  ATC  services.  Section  2  describes  the  specific 
objectives  of  the  study  and  the  testing  approach  that  was  used  to 
achieve  these  objectives.  Section  3  presents  the  detailed  results 
of  the  testing.  Sections  4  and  5  list  the  conclusions  that  were 
derived  from  the  results  and  recommendations  for  future  efforts 
toward  the  development  of  an  operational  terminal  Data  Link  system. 

2.  TEST  DESCRIPTION. 

2.1  OBJECTIVES. 

The  Data  Link  mini  study  reported  here  was  designed  to  meet  three 
major  objectives: 

a.  Determine  the  acceptability  of  the  preliminary  designs  for 
Data  Link  terminal  services  as  implemented  in  NAS  (ARTS  IIIA) 
equipment  and  software. 

As  noted  in  the  introduction  to  this  document,  the  four 
designs  of  the  services  tested  in  the  study  were  defined  by  the 
terminal  controller  members  of  the  ATDLVT,  and  reviewed  using  a 
dynamic  prototyping  system.  For  the  present  study,  modified 
versions  of  these  designs  were  transferred  to  the  operational  ARTS 
IIIA  system  maintained  in  the  Data  Link  Test  Bed.  Because  the 
rapid  prototype  system  manual  input  and  visual  display  devices 
differed  significantly  from  the  ARTS  IIIA  user  interface,  several 
design  changes  were  made  to  complete  the  transition.  In  addition, 
selected  display  features  including  the  transaction  list  and  the 
history  list  were  not  implemented  in  the  initial  ARTS  IIIA  versions 
of  the  services.  Finally,  potential  enhancements  were  added  to  the 
basic  designs  for  evaluation  by  the  test  controllers. 

b.  Identify  requirements  for  additional  Data  Link  service 
design  modifications  and  future  controller  testing. 

Past  experience  with  controller  testing  of  en  route  Data 
Link  services  has  shown  that  an  iterative  evaluation  process  is 
necessary  to  produce  designs  which  are  sufficiently  robust  for 
operational  testing.  Because  of  this,  the  second  objective  of  the 
present  study  was  to  provide  additional  simulation  experiences  to 
the  ATDLVT  which  would  elicit  any  required  modifications  to  the 
original  designs  that  had  not  been  foreseen  during  conceptual 
development. 

c.  Elicit  controller  estimates  of  the  operational  suitability, 
user  acceptance,  and  workload  impact  of  Data  Link  services  in  the 
terminal  ATC  environment. 
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The  final  objective  of  the  research  was  to  obtain  an 
assessment  of  whether  Data  Link  ATC  services  continue  to  appear 
feasible  in  the  context  of  terminal  operations.  Based  on  formal 
simulation  experiences  with  the  initial  services  as  implemented  in 
the  Data  Link  Test  Bed,  the  controller  subjects  provided  projective 
expert  input  regarding  the  effects  of  Data  Link  on  ATC  operations 
and  controller  performance  capabilities. 

2.2  APPROACH. 

The  approach  that  was  adopted  to  meet  the  objectives  of  this  mini 
study  involved  the  participation  of  terminal  air  traffic 
controllers  in  a  series  of  training  and  ATC  simulation  test 
sessions  presented  at  the  ARTS  II1A  workstations  in  the  Data  Link 
Test  Bed.  During  testing,  the  subjects  controlled  traffic  in  a 
scenario  involving  aircraft  arrivals  and  final  approaches.  During 
the  early  test  sessions,  the  subjects  completed  a  detailed  review 
of  the  designs  for  each  of  the  four  Data  Link  services.  In  the 
final  repetitions  of  the  scenario,  the  subjects  compared  the 
current  voice-only  communication  system  to  a  system  supplemented 
by  the  initial  Data  Link  services  in  order  to  make  predictive 
judgments  about  the  effects  of  Data  Link  on  the  ATC  system  and 
controller  performance  factors. 

Extensive  debriefings  followed  each  test  session.  These 
debriefings  were  used  to  achieve  a  group  consensus  regarding 
required  Data  Link  service  design  changes  and  to  elicit  controller 
opinions  regarding  future  testing  needs. 

The  general  rationale  underlying  the  test  design  was  to  provide  a 
relatively  uncomplicated  ATC  environment  which  would  permit  the 
controllers  to  devote  their  attention  to  an  exhaustive  analysis  of 
the  service  designs,  while  providing  sufficient  realism  to  support 
their  initial  projections  regarding  the  impact  of  the  services 
under  operational  conditions.  Highly  structured  questionnaires  and 
quantified  rating  scales  were  used  as  primary  assessment  tools  in 
order  to  provide  an  objective  baseline  against  which  future 
modifications  to  terminal  Data  Link  functionality  could  be  gauged. 

2.3  TEST  CONDUCT. 


2.3.1  Subjects. 

The  subjects  for  this  study  were  nine,  full  performance  level, 
terminal  air  traffic  control  specialists.  All  subjects  were 
members  of  the  FAA  ATDLVT  which  was  formed  to  participate  in  the 
development  and  evaluation  of  Data  Link  ATC  services.  FAA  field 
facilities  represented  by  the  subjects  were  Denver,  Madison  (WI) , 
St.  Louis,  and  Tulsa  Airport  Traffic  Control  Towers  ( ATCTs)  and 
Ocean  (NY) ,  and  Ontario  (CA)  Terminal  Radar  Approach  Control 
Facilities  (TRACONs) . 
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Seven  of  the  nine  subjects  had  participated  in  a  majority  of  all 
conceptual  design  exercises  held  during  a  2-year  period  prior  to 
the  study.  The  remaining  two  subjects  were  new  members  of  the 
ATDLVT  who  received  their  first  experience  with  Data  Link  during 
the  course  of  the  study. 

2.3.2  Test  Scenario  and  Data  Link  Operations. 

The  ATC  scenario  developed  for  this  study  utilized  the  Atlantic 
City  International  Airport  (ACY),  NJ,  Approach  Control  Airspace, 
(figure  3).  For  the  purposes  of  the  study,  several  modifications 
were  made  to  the  actual  sectorization  of  the  surrounding  airspace 
and  to  the  air  traffic  procedures  used  at  the  facility.  Within  the 
test  scenario,  ACY  controlled  the  airspace  vertically  from  surface 
to  8,000  feet.  All  existing  restricted  and  warning  areas  were 
considered  inactive,  and  the  extensive  airspace  shelves  to  the 
north  shared  with  Washington  and  New  York  Air  Route  Traffic  Control 
Centers  (ARTCCs)  were  eliminated  and  considered  part  of  ACY 
approach.  Runway  31  was  active  throughout  the  scenario. 

The  ACY  approach  airspace  was  divided  into  three  sectors  manned  by 
alternating  subgroups  of  three  test  subjects.  The  North  Arrival 
sector  included  all  airspace  north  of  the  active  runway  excluding 
the  Final  airspace.  The  South  Arrival  sector  included  all  airspace 
south  of  the  runway  excluding  the  Final  airspace.  The  Final 
controller  was  responsible  for  the  area  5  miles  to  either  side  of 
the  final  approach  course,  from  the  airport  to  15  miles,  and  from 
surface  to  3,000  feet. 

As  shown  in  figure  3,  inbound  traffic  was  transferred  to  the  North 
and  South  sectors  over  four  Transfer  of  Control  Points.  Both  the 
North  and  South  controllers  were  required  to  blend  two  streams  of 
aircraft  and  establish  each  aircraft  on  downwind  for  runway  31  at 
4,000  feet.  After  any  conflicts  had  been  resolved,  the  North  and 
South  controllers  handed  off  the  aircraft  to  the  Final  controller. 
The  Final  controller  then  sequenced  the  two  streams  of  traffic  onto 
the  final  and  cleared  each  aircraft  for  approach. 

During  each  30-minute  repetition  of  the  scenario,  the  North  and 
South  controllers  each  handled  approximately  13  aircraft,  while  the 
Final  controller  was  responsible  for  all  arriving  traffic. 

Pilot  functions  for  this  study  were  provided  using  the  ARTS 
Enhanced  Target  Generator  ( ETG)  function.  During  each  test  run, 
three  of  the  subjects  manned  the  active  control  sectors.  The 
remaining  six  subjects  acted  as  simulation  pilots  using  three 
additional  ARTS  workstations.  Two  simulation  pilots  were  seated 
at  each  workstation.  One  of  the  pilots  maintained  voice  radio 
communication  while  the  other  used  a  terminal  interfaced  to  the 
VAX  computer  to  respond  to  Data  Link  communications.  Both  pilots 
made  ETG  inputs  to  control  the  flightpaths  of  the  aircraft. 
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When  using  Data  Link  for  ATC  communications,  the  elapsed  time 
between  a  message  uplink  command  and  receipt  of  a  pilot  initiated 
response  was  determined  by  the  simulator  pilot's  response 
latency.  This  includes  an  uplink  and  downlink  time  which  was 
introduced  to  simulate  the  delays  that  would  be  experienced  with 
Mode  S  Data  Link  transmissions  using  the  currently  operational 
terminal  antenna  rotation  rate  of  4  seconds.  Uplink  delays  ranged 
from  0  to  4  seconds.  Downlink  delays  were  determined  by  the  timing 
of  the  simulator  pilot's  response,  increasing  from  0  in  4-second 
increments . 

In  order  to  exercise  controller  procedures  for  handling  failed  Data 
Link  transmissions,  10  percent  of  all  uplink  attempts  resulted  in 
no  technical  acknowledgement  (NAK)  by  the  simulated  aircraft  Data 
Link  avionics.  Likewise,  if  a  simulator  pilot  failed  to  respond 
to  a  Data  Link  message  within  40  seconds  after  receipt,  a  time¬ 
out  message  was  sent  to  the  ARTS.  In  accordance  with  the 
preliminary  service  design  specifications,  either  a  NAK  or  a  time¬ 
out  resulted  in  a  FAIL  message  in  the  aircraft's  data  block  on  the 
controller's  radar  display. 

2.3.3  Test  Procedures. 

This  study  was  conducted  over  a  3-day  period  in  conjunction  with 
a  scheduled  meeting  of  the  full  membership  of  the  ATDLVT  held  at 
the  FAA  Technical  Center.  The  first  day  was  devoted  to  subject 
prebriefing  and  training.  On  the  second  and  third  days,  testing 
was  conducted  to  support  a  review  of  the  Data  Link  services  designs 
and  projective  evaluation  of  the  operational  impact  of  Data  Link. 

2.3.3. 1  Familiarization  and  Training. 

The  prebriefing  session  began  with  a  series  of  short  presentations 
which  were  used  to  explain  the  objectives  of  the  mini  study  and  the 
testing  activities.  The  subjects  also  received  a  briefing  on  the 
questionnaires  and  rating  scales  that  would  be  used  during  data 
collection,  and  completed  the  scale  development  card  sorting  task 
required  for  the  interpretation  of  workload  judgments  using  the 
Projective  Subjective  Workload  Assessment  Technique  (PROSWAT)  (see 
section  2.3.4)  . 

Familiarization  with  the  Data  Link  service  designs  was  accomplished 
in  a  1-hour  classroom  session  during  which  all  data  input 
procedures  and  displays  were  explained.  Differences  between  the 
ARTS  IIIA  Test  Bed  implementation  of  the  services  and  the 
preliminary  design  specifications  also  were  discussed  during  this 
session.  Finally,  the  subjects  received  a  briefing  on  the  airspace 
scenario,  the  ATC  procedures  that  would  be  used  during  testing,  and 
on  the  operation  of  the  ETG  simulation  system. 

Following  the  prebriefings,  the  subjects  proceeded  to  the  Data  Link 
Test  Bed  for  hands-on  training.  Subjects  were  assigned  to  a  sector 
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in  one  of  the  three  subgroups  and  were  given  the  opportunity  to 
control  traffic  with  several  30-minute  test  scenarios.  In  order 
to  maximize  the  efficiency  of  training,  all  aircraft  in  the 
scenario  were  equipped  with  Data  Link  during  this  session. 
Facilitators  were  available  at  each  sector  to  assist  the  subjects 
and  answer  questions. 

2. 3. 3. 2  Design  Review. 

Data  collection  began  on  the  second  day  of  the  study  with  a  formal 
review  and  verification  of  the  Data  Link  service  designs.  The 
design  review  was  accomplished  in  the  Data  Link  Test  Bed.  Each 
subgroup  of  three  subjects  controlled  traffic  in  the  scenario 
during  two  30-minutc  runs.  The  primary  task  of  each  subject  during 
these  runs  was  to  thoroughly  exercise  each  of  the  four  Data  Link 
service  designs  in  order  to  evaluate  them  using  a  structured  review 
booklet . 

The  review  booklet  was  organized  in  two  sections.  The  first 
focused  on  the  primary  differences  between  the  ARTS  IIIA 
implementation  of  the  Data  Link  services  and  the  specifications 
developed  by  the  ATDLVT  during  earlier  conceptual  design  efforts. 
These  differences  included  the  absence  of  a  history  list  of 
completed  Data  Link  transactions  in  the  tested  ARTS  configurations, 
and  of  a  list  which  tracked  the  status  of  transactions  in  progress. 
The  first  section  also  addressed  differences  in  keyboard 
assignments  for  Data  Link  inputs  and  in  the  graphic  symbols  used 
in  the  data  block  to  denote  Data  Link  equipage  and  the  controller's 
eligibility  to  communicate  with  the  aircraft.  Associated 
questionnaire  items  asked  the  subjects  to  evaluate  the  need  for  the 
display  features  not  included  in  the  tested  ARTS  versions,  and  the 
acceptability  of  the  display  substitutions  that  were  made. 

The  second  section  of  the  design  review  addressed  each  of  the 
service  designs  in  detail.  The  subjects  were  given  a  text 
description  of  the  operational  features  of  each  service  (see 
appendix  B)  ,  and  were  first  asked  to  judge  the  accuracy  of  the 
description  against  their  actual  experience  in  the  Test  Bed. 
Succeeding  questions  required  them  to  judge  the  similarity  of  the 
ARTS  implementation  to  the  original  design  specification,  the 
acceptability  of  the  design  as  presented  in  the  Test  Bed,  and  the 
need  for  future  modifications  to  the  Test  Bed  service  designs.  For 
each  service,  the  subjects  were  asked  to  fully  describe  any  design 
changes  that  were  either  desirable  or  required. 

After  all  subjects  had  completed  the  Test  Bed  exercise  and  the 
individual  design  review  booklets,  they  participated  in  a  group 
debriefing  session.  This  3-hour  session  was  led  by  a  research 
staff  member  and  consisted  of  a  group  discussion  of  each  service 
design.  The  debriefing  was  aimed  at  identifying  and  resolving 
disagreements  among  the  subjects  regarding  the  fidelity  and 
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acceptability  of  the  service  designs,  and  at  achieving  a  consensus 
regarding  any  recommended  design  changes. 

2.J.3.3  Projective  Evaluations . 

The  second  day  of  testing  reguired  each  subgroup  of  three 
controllers  to  participate  in  two  simulation  runs  of  the  test 
scenario.  In  one  of  these  runs,  the  subjects  completed  their  ATC 
tasks  using  only  the  voice  radio  system.  In  the  second  run,  60 
percent  of  the  aircraft  in  the  scenario  were  equipped  for  Data  Link 
communications  in  order  to  simulate  a  relatively  mature  stage  of 
system  implementation.  To  minimize  testing  sequence  bias  in  the 
subsequent  rating  tasks,  two  of  the  subgroups  experienced  the 
voice-only  run  prior  to  the  Data  Link  run,  while  the  third  subgroup 
participated  in  the  conditions  in  a  reversed  order.  In  both  cases, 
the  subjects  were  instructed  to  treat  the  ATC  problem  as 
realistically  as  possible  and  to  focus  on  comparing  the  current 
voice  radio  system  to  the  combined  Data  Link  and  voice  system. 

Following  the  completion  of  both  runs,  each  subject  completed  a 
series  of  rating  forms.  The  first  form  asked  subjects  for  two 
workload  ratings.  The  first  of  these  was  an  estimate  of  the 
average  workload  experienced  by  a  controller  on  a  typically  busy 
day  in  the  responder's  home  facility  using  the  current  voice-only 
system.  The  second  rating  asked  the  subjects  to  make  a  comparative 
projective  estimate  of  the  controller's  workload  in  the  same 
situation  with  a  60  percent  level  of  Data  Link  equipage  in  the 
aircraft  fleet. 

After  completing  the  workload  ratings,  the  subjects  evaluated  each 
of  the  four  Data  Link  services  on  two  dimensions.  The  first  of 
these  was  the  projected  operational  effectiveness  and  suitability 
of  the  service,  while  the  second  was  the  acceptability  of  the 
service  to  controllers  and  their  predicted  preference  for  the 
features  of  the  service  design.  For  both  dimensions,  the  subjects 
were  asked  to  base  their  ratings  on  the  modified  ARTS  IIIA  service 
design  baselines  established  during  the  design  review  and 
discussions  conducted  on  the  preceding  test  day.  The  subjects  were 
asked  to  provide  written  comments  and  a  rationale  for  each  of  the 
ratings  that  were  made. 

After  the  subjects  had  completed  the  rating  forms,  a  final  group 
debriefing  was  conducted.  This  debriefing  solicited  any  further 
design  modification  requirements  that  may  have  arisen  during  the 
second  day  of  testing  and  presented  several  additional  issues  for 
discussion.  These  issues  included  the  feasibility  and  desirability 
of  implementing  Data  Link  in  the  existing  ATC  system,  the  potential 
impact  of  Data  Link  transmission  delays  on  controller  performance, 
the  conditions  under  which  Data  Link  and  voice  communications  would 
be  used,  and  future  requirements  for  Data  Link  testing. 
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2.3.4  Data  Collection. 

The  data  obtained  in  this  study  were  collected  using  several 
methods  designed  to  elicit  structured  and,  where  possible, 
quantified  expert  input  from  the  air  traffic  controller  subjects. 
As  described  in  section  2. 3. 3. 2,  the  design  review  was  accomplished 
with  a  series  of  questionnaire  forms  which  guided  the  subjects 
through  an  evaluation  of  the  major  alterations  to  the  prototype 
Data  Link  service  designs  and  provided  them  with  the  opportunity 
to  record  individual  suggestions  for  design  modifications.  The 
object  of  this  exercise  and  the  following  debriefing  was  to 
establish  an  initial  ARTS  IIIA  design  for  Data  Link  services  upon 
which  future  development  will  be  based. 

The  projective  data  generated  on  the  second  day  of  testing  was 
collected  using  three  subjective  rating  instruments.  Workload  was 
estimated  using  a  modification  of  the  Subjective  Workload 
Assessment  Technique  (SWAT)  known  as  Projective  SWAT  or  PROSWAT. 
SWAT  and  PROSWAT  were  developed  in  the  early  1980's  by  the  U.S.  Air 
Force  as  a  standardized  method  for  obtaining  quantified  estimates 
of  perceived  workload  in  a  broad  variety  of  occupational  tasks. 
The  techniques  have  received  extensive  use  in  simulation  and 
operational  testing  environments  within  the  Depart  ent  of  Defense, 
and  were  used  successfully  with  air  traffic  controllers  in  the  en 
route  Data  Link  mini  studies  and  operational  evaluation.  SWAT  is 
used  to  rate  the  actual  workload  experienced  by  a  subject  in  a 
preceding  period  of  work,  while  PROSWAT  is  the  identical  method 
used  during  the  research  and  development  process  to  estimate  the 
workload  that  would  be  experienced  in  a  fully  implemented 
operational  system. 

Briefly,  both  SWAT  and  PROSWAT  consist  of  three,  3-point  rating 
scales  referring  to  the  dimensions  of  time  load,  mental  effort,  and 
psychological  stress.  Subjects  indicate  the  workload  associated 
with  an  activity  by  marking  the  appropriate  point  on  each  scale. 
A  unique  feature  of  this  workload  measurement  technique  is  that  the 
three  ordinal  ratings  that  are  obtained  are  converted  to  single 
points  on  an  overall  interval  measurement  scale  of  workload  with 
values  ranging  from  0  (low  workload)  to  100  (high  workload)  using 
a  mathematical  analysis  method  known  as  conjoint  measurement.  This 
method  not  only  yields  data  which  are  amenable  to  powerful, 
parametric  tatistical  testing,  but  also  tailors  the  measurement 
scale  to  each  individual's  (or  homogeneous  group's)  concept  of  how 
the  time,  effort  and  stress  dimensions  combine  to  produce  the 
overall  perception  of  workload. 

The  interval  scale  used  to  interpret  the  ordinal  ratings  is  created 
by  having  subjects  generate  an  ordering  of  all  27  combinations  of 
the  time,  effort,  and  stress  levels  on  the  scales  which  reflects 
their  individual  concepts  of  how  the  three  dimensions  combine  to 
produce  different  workload  levels.  The  card  sorting  task  used 
during  the  scale  development  exercise  was  completed  by  all  nine 
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subjects  in  this  study  during  the  prebriefing  session  held  on  the 
first  day.  PP.OSWAT  ratings  of  controller  workload  comparing  the 
current  voice-only  system  with  an  ATC  system  supplemented  by  Data 
Link  were  made  by  the  subjects  following  the  simulation  runs  on  the 
final  test  day. 

After  completing  their  PROSWAT  ratings,  the  subjects  rated  each  of 
the  four  terminal  services  on  two  factors.  The  first  scale 
required  the  subjects  to  assess  the  operational  effectiveness  and 
suitability  of  the  service  design.  The  data  form  permitted  the 
subject  to  rate  a  service  as  "not  operationally  suitable"  or  on  a 
7-point  scale  ranging  from  1  "minimally  effective"  to  7  "highly 
effective."  Instructions  for  completing  this  scale  asked  the 
subjects  to  use  their  experience  in  the  Test  Bed  and  their  prior 
background  in  terminal  ATC  operations  to  assess  how  well  each 
service  design  could  accomplish  its  intended  task  in  the  full  range 
of  potential  field  environments. 

The  second  rating  was  an  estimate  of  the  acceptability  and 
preferability  of  each  of  the  service  designs  to  air  traffic 
controllers.  The  data  form  permitted  the  subjects  to  rate  a 
service  as  "completely  unacceptable"  or  on  a  7-point  scale  ranging 
from  1  "acceptable,  but  not  preferred"  to  7  "highly  preferred." 
Instructions  directed  the  subjects  to  consider  the  extent  to  which 
the  displays,  input  requirements,  and  procedures  used  to  deliver 
a  service  would  be  usable  by  air  traffic  controllers. 

Additional  explicit  instructions  for  these  two  scales  indicated 
that  the  dimensions  of  effectiveness  and  preference  should  be 
treated  independently  from  one  another  since  a  service  might 
include  all  functions  needed  to  meet  operational  requirements  and 
still  be  poorly  suited  to  the  way  in  which  controllers  perform 
their  functions.  Likewise,  a  design  could  be  easy  to  use,  but  have 
missing  features  which  prevent  it  from  meeting  operational  needs. 

The  instructions  provided  to  subjects  for  the  use  of  all  rating 
scales  and  for  the  SWAT  scale  development  task  are  included  in 
appendix  C  along  with  examples  of  the  actual  rating  forms. 

3.  TEST  RESULTS. 

3.1  TERMINAL  DATA  LINK  SERVICE  DESIGN  REVIEW. 

Two  major  objectives  of  the  present  study  were  to  determine  the 
acceptability  of  the  Data  Link  service  designs  as  implemented  in 
the  ARTS  IIIA  portion  of  the  Data  Link  test  bed,  and  to  obtain 
controller  inputs  regarding  desirable  modifications  to  these 
designs.  The  following  subsections  summarize  the  combined  results 
obtained  from  the  individual  design  review  booklets  and  from  the 
debriefing  discussion  that  was  conducted  following  the  review 
exercise . 
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3.1.1  Transaction  Status  List. 


The  preliminary  design  for  the  terminal  services  provided 
information  regarding  the  status  of  ongoing  Data  Link  transactions 
both  in  abbreviated  form  in  the  data  block  associated  with  each 
equipped  aircraft,  and  in  a  combined  list  form.  The  status  list 
was  not  included  in  the  ARTS  IIIA  implementation  tested  in  this 
study . 

Individual  subject  responses  regarding  the  need  for  a  status  list 
were  mixed.  Of  the  seven  subjects  who  had  participated  in  the 
conceptual  design  effort  prior  to  the  study  and  had  worked  with 
prototype  data  block  and  status  list  displays,  three  felt  that  the 
status  list  was  unimportant,  two  indicated  that  it  was  a  preferred 
but  not  a  necessary  feature  of  the  design,  and  two  indicated  that 
it  would  be  required.  Those  subjects  who  felt  this  feature  was 
unimportant  noted  that  it  was  redundant  with  the  primary  data  block 
display  of  the  menu  item  number  sent  and  the  outcome  of  the 
transaction.  The  subjects  expressing  some  level  of  preference  for 
the  list  felt  that  while  the  data  block  is  a  primary  display,  any 
requirement  to  quickly  determine  the  content  of  the  transaction  in 
progress  would  be  hampered  by  the  need  to  scan  to  the  menu  text 
list  to  interpret  the  menu  item  number  in  the  data  block.  In  these 
cases,  the  list's  inclusion  of  the  full  content  of  a  clearance 
would  simplify  the  task  of  rechecking  the  message. 

Group  discussion  of  the  status  list  led  to  a  consensus  opinion  that 
the  status  list  should  be  included  in  the  ARTS  IIIA  Data  Link 
design  for  further  testing.  In  addition,  it  was  determined  that 
the  status  list,  as  well  as  the  data  block  display  of  transaction 
status,  be  modified  to  include  more  extensive  information  (see 
section  3.1.2  below). 

3.1.2  Data  Block  Transaction  Status  Display. 

During  conceptual  development  of  the  Data  Link  ATC  services,  a 
majority  of  both  terminal  and  en  route  members  of  the  ATDLVT  have 
argued  that  the  controller  should  not  be  required  to  divert 
attention  from  the  aircraft  on  the  radar  display  in  order  to 
monitor  the  status  of  ongoing  Data  Link  transactions.  In  the 
terminal  effort,  this  led  to  the  use  of  the  third  line  of  the  data 
block  as  a  transaction  display.  However,  because  the  controllers 
expressed  an  equally  strong  belief  that  information  inserted  in  the 
data  block  must  be  simplified  to  prevent  display  clutter  and 
controller  confusion,  the  data  presented  in  the  third  line  were 
limited  to  an  abbreviation  of  the  service  in  progress,  and  an 
alerting  display  if  the  transaction  was  not  satisfactorily 
completed.  No  indication  that  an  uplink  had  been  successfully  sent 
or  delivered  to  the  aircraft  was  included.  Likewise,  only  removal 
of  the  transaction  data  from  the  third  line  was  used  to  indicate 
that  a  pilot  had  confirmed  message  receipt  and  intended  to  comply. 
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and  no  distinction  was  made  among  the  several  reasons  for 
transaction  failure. 

During  the  design  review,  the  subject  controllers  noted  that  more 
complete  status  information  may  be  needed  in  the  data  block  and  in 
the  status  display  to  avoid  potential  errors  and  to  permit  the 
controller  to  appropriately  deal  with  a  failed  transaction.  Two 
potential  sources  of  error  were  cited  in  support  of  this 
requirement.  First,  removal  of  the  data  in  the  third  line  as  a 
means  for  indicating  that  a  message  had  been  WILCOed  was  considered 
a  problem  since  it  presents  the  opportunity  for  a  busy  controller 
to  forget  whether  the  message  had  been  originally  uplinked.  One 
solution  to  this  problem  that  was  suggested  is  to  display  A  "W" 
character  or  the  word  WILCO  when  the  pilot  responds  positively,  and 
for  this  data  to  remain  in  the  status  list  and  the  third  line  of 
the  data  block  for  several  seconds  or  until  a  new  transaction  is 
initiated. 

The  lack  of  a  displayed  distinction  between  causes  for  transaction 
failures  was  seen  as  second  possible  source  of  problems.  The 
controllers  noted  that  their  decision  to  respond  to  a  failed 
transaction  by  repeating  the  uplink  or  by  contacting  the  pilot  by 
voice  radio  may  depend  upon  whether  the  failure  was  caused  by  a 
technical  error  in  the  digital  transmission,  a  pilot's  failure  to 
respond  to  a  message  within  the  prescribed  time  limits,  or  a  pilot 
downlink  indicating  that  he  was  unable  to  comply  with  the  uplinked 
clearance.  As  a  result  of  this  discussion,  it  was  decided  that  the 
ARTS  Test  Bed  Data  Link  design  would  be  modified  to  present  full 
failure  information  in  the  data  block  and  in  the  status  list. 

3.1.3  Transaction  History  List. 

The  prototype  design  for  the  terminal  services  included  a  second 
list  which  displayed  messages  which  had  been  successfully  uplinked 
to  each  aircraft  in  the  controller's  sector.  This  feature  was  not 
included  in  the  ARTS  implementation  evaluated  during  this  study. 
The  results  of  the  design  review  indicated  that  although  one 
subject  felt  that  the  history  list  was  not  an  important  feature  of 
the  services.  Two  indicated  that  the  list  was  a  necessary  feature, 
and  five  felt  that  it  was  preferable,  but  not  necessary.  The 
majority  preference  for  the  history  list  was  supported  by 
individual  comments  suggesting  that  it  may  be  valuable  as  a 
general  memory  aid  which  would  prevent  repeated  transmissions  and 
pilot  inquiries.  During  debriefing,  it  was  decided  that  the 
history  list  would  be  added  to  the  ARTS  Test  Bed  implementation  of 
Data  Link  for  future  evaluation. 

Subsequent  discussion  of  the  optimal  number  of  items  to  be  included 
in  the  list  did  not  result  in  a  consensus  opinion.  Requests  ranged 
from  only  the  last  clearance  received  by  each  aircraft  up  to  a 
total  record  of  the  messages  received  by  an  aircraft  while  in  the 
sector.  Arguments  for  lists  containing  only  the  last  completed 
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total  record  of  the  messages  received  by  an  aircraft  while  in  the 
sector.  Arguments  for  lists  containing  only  the  last  completed 
uplink  indicated  that  the  list  would  be  used  primarily  for 
resolving  momentary  lapses  of  short  term  memory,  while  subjects 
preferring  longer  lists  felt  that  it  would  be  necessary  to  capture 
sufficient  past  messages  to  determine  an  aircraft's  current 
assigned  heading,  speed,  and  altitude.  As  a  result  of  this  split 
opinion,  list  length  will  be  empirically  investigated  in  a  future 
mini  study. 

3.1.4  Data  Link  Equipage  and  Eligibility  Display. 

The  basic  design  for  both  en  route  and  terminal  Data  Link  services 
requires  a  data  block  display  indicating  whether  the  aircraft  is 
equipped  for  Data  Link  communications,  and  which  controller  is 
eligible  to  communicate  with  the  aircraft.  In  both  the  HOST/plan 
view  display  (PVD)  en  route  system  and  the  terminal  designs,  this 
display  is  provided  by  graphic  symbology  located  in  the  first 
position  of  the  first  line  of  the  data  block.  The  prototype 
terminal  design  used  a  plus  sign  to  denote  an  equipped  aircraft 
with  which  the  current  controller  was  eligible  to  communicate,  and 
an  asterisk  to  identify  an  equipped  aircraft  which  was  eligible  to 
receive  uplinks  from  the  current  controller.  Because  of  concerns 
about  the  legibility  and  distinguishability  of  these  symbols  on  the 
ARTS  IIIA  radar  display,  an  open  square  and  open  triangle  were  used 
in  the  initial  Test  Bed  implementation  to  provide  the  same 
information. 

In  the  design  review,  none  of  the  subjects  indicated  that  the 
information  offered  by  this  symbology  was  inadequate  and  three 
controllers  were  completely  satisfied  by  the  use  of  the  open 
geometric  shapes.  However,  six  subjects  felt  that  the  original 
symbology  would  be  easier  to  use.  One  argument  for  this  preference 
was  that  the  square  could  be  misperceived  as  an  "0"  used  in 
military  call  signs. 

During  the  debriefing  session,  the  general  problem  of  symbol 
confusability  with  other  ARTS  symbol  assignments  and  the 
exacerbation  of  this  problem  caused  by  display  misalignment  led  the 
group  to  suggest  that  alternate  symbol  sets  be  maintained  as  a 
variable  feature  of  the  Test  Bed  for  review  in  subsequent  research. 

3.1.5  Keyboard  Assignments. 

The  prototype  demonstration  of  the  conceptual  terminal  Data  Link 
service  designs  used  a  generic,  personal  computer-type  keyboard  for 
Data  Link  inputs.  Because  of  this,  implementation  on  the 
idiosyncratic  keyboard  used  in  the  ARTS  system  required  changes  in 
the  location  of  the  keys.  Eight  of  the  nine  subjects  found  the 
assignments  used  in  the  ARTS  implementation  acceptable.  A 
dissenting  controller  felt  that  the  location  of  the  Data  Link  key 
in  the  F9  position  in  the  upper  left  quadrant  of  the  keyboard  could 
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be  somewhat  inconvenient  because  right  handed  users  would  obscure 
most  of  the  keyboard  when  pressing  this  key.  Changing  the 
assignment  to  the  right  hand  extreme  of  the  function  key  row  was 
suggested  as  an  alternative,  and  this  option  will  be  included  for 
testing  in  future  studies. 

Debriefing  discussions  of  the  keyboard  and  general  input  procedures 
revealed  a  potential  dilemma  in  Data  Link  integration  with  an 
operational  system.  While  there  was  strong  agreement  that  input 
sequences  should  be  used  which  minimize  the  number  of  keystrokes 
required  to  initiate  an  uplink,  the  use  of  implied  functions  to 
accomplish  this  goal  was  seen  as  a  potential  source  of  controller 
error.  Using  the  implied  function  features  included  in  the 
original  design  and  retained  in  the  tested  ARTS  implementation,  it 
is  possible  to  initiate  most  uplinks  without  preceding  the  data 
command  by  the  Data  Link  key  input.  While  reducing  keystrokes, 
this  feature  creates  some  inconsistencies  which  require  the 
controller  to  remember  when  the  Data  Link  key  input  is  required. 
In  addition,  the  existing  use  of  implied  functions  for  other  ARTS 
functions  makes  it  possible  to  initiate  an  unwanted  function  when 
an  error  is  made  in  a  Data  Link  entry. 

No  final  design  consensus  was  reached  on  this  issue.  It  was 
determined  that  future  tests  should  be  conducted  in  which  keyboard 
entries  are  recorded  in  order  to  identify  and  remediate  any  common 
keyboard  entry  errors  that  arise. 

3.1.6  Initial  Contact  Service. 

All  of  the  subjects  who  had  participated  in  the  conceptual 
development  process  agreed  that  the  Test  Bed  implementation 
faithfully  reflected  the  original  design  specification  and  that  the 
design  was  equal  to,  or  better  than,  the  prototype  demonstration. 
All  nine  subjects  also  indicated  that  no  modifications  to  this 
service  were  required. 

3.1.7  Terminal  Information  Service. 

Six  of  the  seven  subjects  who  participated  in  conceptual 
development  indicated  that  the  ARTS  implementation  of  the  terminal 
information  (TI)  service  was  equivalent  to,  or  better  than  the 
prototype  implementation.  Of  the  full  subject  sample,  four 
subjects  indicated  that  no  design  changes  were  necessary.  However, 
the  remaining  five  subjects  indicated  that  improvements  were 
needed.  In  particular,  these  subjects  felt  that  the  option  to 
combine  a  TI  message  with  one  or  more  menu  text  messages  should  be 
enhanced  to  permit  the  use  of  any  of  the  TI  messages  rather  than 
limiting  this  option  to  the  default  message. 

The  subjects  also  noted  the  requirement  for  a  simplified  method  for 
changing  the  content  of  the  default  message.  The  default  message 
feature  was  developed  to  permit  simplified  uplink  of  the  most 
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commonly  used  information  at  a  sector.  In  order  to  accommodate 
changes  in  this  message,  the  tested  design  permitted  the  controller 
to  retype  a  new  message  in  the  default  position  of  the  list. 
Subjects  who  requested  a  change  in  this  feature  preferred  an  option 
to  make  an  entry  which  would  shift  an  existing  message  into  the 
default  position  of  the  TI  list.  Both  modifications  were  discussed 
in  the  debriefing  session  and  adopted  as  Test  Bed  changes.  In  the 
modified  version,  TI  list  items  will  be  numbered  from  1  to  N,  with 
the  default  message  denoted  by  its  position  at  the  top  of  the  list. 
Selection  of  a  new  default  item  will  move  the  item  to  the  top 
position  and  retain  its  original  number  designation.  The  remaining 
items  will  then  placed  in  succeeding  positions  determined  by  their 
numerical  designations. 

3.1.8  Transfer  of  Communication. 

Two  subjects  indicated  that  no  changes  to  the  design  for  the 
transfer  of  communication  (TC)  service  were  needed,  while  the 
remaining  seven  noted  changes  that  were  either  desirable  or 
necessary.  The  most  commonly  required  modification  mentioned  was 
the  inclusion  of  an  ability  to  maintain  Data  Link  eligibility  in 
the  interim  period  between  the  completion  of  the  sector  handoff 
and  the  initiation  of  the  radio  frequency  uplink  through  a  manual 
input.  The  tested  ARTS  design  did  not  permit  an  uplink  while  the 
TC  HELD  message  was  displayed  in  the  third  line  of  the  data  block. 
This  modification  is  required  to  accommodate  the  common  controller 
practice  of  completing  the  handoff  well  in  advance  of  an  aircraft 
exiting  the  sector. 

Permitting  other  uplinks  while  a  TC  message  is  in  the  held  state 
will  require  simultaneous  use  of  the  third  line  of  the  data  block 
by  the  two  services.  Future  studies  will  compare  a  timesharing 
approach  to  a  display  in  which  initiation  of  an  uplink  will  replace 
the  TC  HELD  message  until  the  transaction  is  complete. 

3.1.9  Menu  Text. 

All  of  the  subjects  rated  the  ARTS  implementation  of  the  menu  text 
function  equal  to,  or  better  than  the  original  design 
specification.  Unlike  the  original  design,  the  ARTS  implementation 
included  a  menu  by-pass  feature  which  permitted  the  subjects  to 
uplink  a  speed,  heading  or  altitude  by  typing  the  F9  key,  then  an 
S,  H,  or  A  followed  by  the  three-digit  numerical  parameter  for  the 
clearance.  In  the  individual  reviews,  five  subjects  specifically 
commented  that  the  by-pass  function  was  a  definite  enhancement  to 
the  menu  text  service.  Subsequent  debriefing  discussions  revealed 
a  split  preference  for  the  use  of  the  by-pass  function  and  the 
standard  menu  text  for  uplinking  clearances.  While  some  subjects 
chose  to  work  exclusively  with  clearances  entered  as  menu  items, 
others  felt  that  this  restriction  would  create  lengthy  menus  in 
which  the  selection  of  a  clearance  would  become  a  difficult  task. 
These  individuals  felt  that  the  menu  should  be  limited  to  the  most 
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commonly  used  messages  and  that  other  clearances  should  be  uplinked 
with  the  by-pass  function.  To  accommodate  both  preferences,  the 
menu  by-pass  feature  will  be  retained  along  with  the  menu 
selections. 

Suggested  chariges  to  the  tested  service  which  were  agreed  upon  in 
the  group  discussion  included  the  addition  of  single  menu  items 
which  combine  a  speed,  heading,  and  altitude  clearance  in  order  to 
reduce  the  number  of  keystrokes  needed  to  uplink  multiple  messages. 
It  was  also  agreed  that  the  by-pass  feature  be  modified  to  permit 
transmission  of  combined  clearances. 

The  problem  of  selecting  messages  from  potentially  lengthy  menus 
was  treated  in  detail  during  the  debriefing.  While  some  subjects 
saw  the  by-pass  as  a  partial  solution  to  this  problem,  several 
suggestions  were  recorded  for  improving  the  controller's  ability 
to  make  rapid  and  accurate  menu  selections.  These  included  a 
modification  to  permit  spatial  separation  of  the  menu  text  and 
terminal  information  lists  on  the  radar  display,  limitation  of  the 
menu  to  ten  items  in  order  to  eliminate  the  space  bar  keystroke 
between  multiple  menu  selections,  and  the  assignment  of  menu  item 
number  designators  to  clearances  in  a  manner  which  would  promote 
the  ability  to  memorize  needed  items  (e.q. ,  using  item  number  5  for 
an  altitude  clearance  of  5,000  feet). 

It  was  determined  that  the  ARTS  implementation  would  be  modified 
to  permit  separation  of  the  lists.  However,  the  controllers  chose 
not  to  limit  menu  length  until  operational  requirements  are  better 
understood.  Since  the  existing  design  provides  controllers  with  a 
limited  ability  to  assign  clearances  to  different  item  designators, 
the  option  to  examine  the  use  of  mnemonic  techniques  in  menu 
designs  will  be  exercised  in  a  future  study.  Regardless  of  the 
solutions  which  are  adopted,  continued  research  attention  will  be 
given  to  the  issue  of  optimizing  menu  accessibility. 

3 . 2  PROJECTIVE  EVALUATIONS . 


3.2.1  Controller  Workload. 

In  the  present  study,  the  PROSWAT  scaling  and  rating  technique  was 
used  to  obtain  comparative  estimates  of  the  operational  impact  of 
the  tested  Data  Link  services  on  controller  workload  in  the 
terminal  environment.  The  PROSWAT  scale  development  task  performed 
prior  to  data  collection  yielded  card  orderings  for  each  subject 
that  were  subjected  to  the  computerized  SWAT  scaling  routines.  A 
Kendall's  Coefficient  of  Concordance  computed  on  the  card  sorts 
produced  an  index  of  association  among  the  subjects'  card  orderings 
of  .77.  While  this  level  of  agreement  would  be  acceptable  for  the 
development  of  a  group  scale,  a  prototype  analysis  was  performed 
in  order  to  determine  whether  meaningfully  homogeneous  subgroups 
could  be  identified.  This  analysis  indicated  that  the  sorts  of  six 
of  the  subjects  were  most  consistent  with  a  concept  of  workload  in 
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which  time  factors  weigh  most  importantly,  while  the  remaining 
three  sorts  were  most  consistent  with  a  model  in  which  stress 
factors  more  strongly  drive  the  workload  that  they  experience.  It 
is  of  some  interest  to  note  that  both  of  these  models  of  workload 
appeared  congruent  with  the  personal  observations  of  the  terminal 
controllers  who  often  report  that  it  is  the  shortage  of  time, 
overlap  among  task  requirements,  and  associated  stress  of  the 
situation  which  can  make  their  job  of  tactically  controlling 
aircraft  difficult  to  perform. 

Because  of  the  differences  between  these  two  subgroups,  separate 
PROSWAT  scale  solutions  were  developed  and  used  in  transforming  the 
subjects'  ordinal  ratings  on  the  time  effort  and  stress  scales  to 
single  scores  on  the  0  (low  workload)  to  100  (high  workload  scale. 
Table  1  presents  the  converted  scale  value  assigned  to  each 
possible  PROSWAT  rating  for  the  subjects  in  the  two  prototype 
groups . 

Following  the  Test  Bed  simulation  runs  on  the  second  day  of 
testing,  the  subjects  made  two  workload  estimates.  The  first  was 
an  estimate  of  terminal  controller  workload  on  a  typically  busy  day 
at  the  subject's  home  facility  using  the  current  voice-only 
communication  system.  The  second  was  a  projective  rating  of  the 
same  situation  where  the  available  Data  Link  services  were 
available  to  service  60  percent  of  the  aircraft  handled. 

The  median  raw  time  load  (T)  ,  mental  effort  (E)  and  stress  (S) 
ratings  produced  by  the  subjects  for  the  voice  only  system  was  T=3, 
E=2,  S=2.  Median  ratings  for  the  60  percent  Data  Link  environment 
changed  to  T=2 ,  E=2 ,  S=2 ,  indicating  an  apparent  reduction  in  the 
time  demands  of  the  controllers  task.  This  projected  effect  also 
was  reflected  in  the  overall  workload  scores  transformed  to  reflect 
the  controller's  original  weightings  of  the  importance  of  the  three 
workload  dimensions.  As  shown  in  figure  4,  the  mean  PROSWAT  score 
dropped  from  79.36  in  the  voice  radio  estimate  to  53.02  in  the  Data 
Link  projection.  Using  these  scores  as  a  criterion,  six  of  the  nine 
subjects  predicted  an  overall  reduction  in  workload  with  Data  Link, 
three  subjects  projected  no  change,  and  one  indicated  that  workload 
may  increase. 

Statistical  analysis  of  the  SWAT  scores  indicated  that  the 
projected  mean  difference  was  statistically  significant  (t=  2.59, 
p  =  .032).  It  is  noteworthy  that  although  some  earlier  studies  had 
shown  either  no  impact  or  a  trend  toward  lower  workload  estimates 
with  Data  Link,  the  present  research  yielded  the  first  evidence  for 
a  significant  reduction  in  workload. 

Subject  comments  regarding  their  ratings  were  congruent  with  the 
quantitative  results.  The  six  subjects  who  envisioned  lower 
workload  with  Data  Link  referenced  its  impact  on  reducing  the  time 
needed  to  communicate  with  aircraft,  and,  in  some  cases,  the 
collateral  effects  of  reduced  time  pressure  on  the  effort  and 
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stress  dimensions  of  workload.  Both  of  the  subjects  whose  ratings 
revealed  no  workload  difference  expressed  uncertainty  regarding  the 
final  impact  of  the  trade  off  between  voice  and  keyboard 
activities.  The  single  subject  predicting  an  increase  indicated 
that  stress  and  effort  may  rise  because  of  the  requirement  to 
search  a  menu  for  the  needed  instruction  and  enter  it.  Two  other 
subjects  who  mentioned  the  menu  selection  and  keying  issues  in 
their  written  comments  had  predicted  lower  workload  levels  with 
Data  Link,  noting  that  menu  items  would  be  rapidly  memorized  in  the 
operational  environment  and  that  keystrokes  would  become  automatic 
with  practice. 


TABLE  1.  TRANSFORMED  PROSWAT  SCALE  VALUES  FOR  THE  TIME 
AND  STRESS  PROTOTYPE  SUBGROUPS 


SWAT  Rating 

Combination  Time  Prototype  Stress  Prototype 

T  E  S  Scale  Equivalent  Scale  Equivalent 


111 
112 
113 
12  1 

12  2 

12  3 

13  1 

13  2 

13  3 

2  11 

2  12 

2  13 

2  2  1 

2  2  2 

2  2  3 

2  3  1 

2  3  2 

2  3  3 

3  11 

3  12 

3  13 

3  2  1 

3  2  2 

3  2  3 

3  3  1 

3  3  2 

3  3  3 


0 

7.7 

13.6 

10.4 
18.1 
24.0 

19.2 
26.9 

32.8 

32.7 

40.5 
46.4 

43.2 

50.9 

56.8 
52.0 

59.7 

65.6 

67.2 

74.9 

80.8 

77.6 

85.3 

91.2 

86.2 

94.1 

100.0 


0 

31.3 

54.6 

12.1 

43.4 

66.7 

19.8 

51.1 

74.5 
8.7 

40.0 

63.4 

20.8 

52.1 

75.4 

28.6 
59.9 

83.2 

25.5 

56.8 

80.2 

37.6 

68.9 
92.2 
45.4 

76.6 

100.0 
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FIGURE  4.  PROSWAT  PROJECTED  WORKLOAD  ESTIMATES 


3.2.2  Operational  Effectiveness  and  Controller  Acceptance. 


Upon  completing  the  workload  estimates,  the  subjects  were  asked  to 
predict  operational  effectiveness/ suitability  and  controller 
acceptance/preference  for  each  of  the  initial  terminal  Data  Link 
services.  Figure  5  presents  the  minimum,  maximum,  and  median 
ratings  that  were  obtained  for  each  service.  None  of  the  subjects 
rated  any  of  the  services  as  "not  operationally  suitable"  or  as 
"completely  unacceptable"  to  the  controller  users  of  the  system. 
In  addition,  as  shown  in  the  figure,  all  four  services  received 
high  median  effectiveness  and  controller  ratings.  With  the 
exception  of  Menu  Text,  all  of  the  services  were  rated  by  all 
subjects  at  or  above  the  center  point  of  the  two  scales,  indicating 
a  strong  and  consistently  positive  opinion  of  the  designs  as  tested 
and  modified  during  this  study.  A  single  subject  rated  the  Menu 
Text  function  below  the  midpoint  of  both  scales  (1  -  minimally 
effective/acceptable,  but  not  preferred).  This  subject's  comments 
referenced  the  difficulty  in  rapidly  locating  desired  menu  items 
discussed  earlier  in  the  results.  Two  other  subjects  expressed 
similar  concerns  in  their  comments  about  this  service  and 
emphasized  that  additional  testing  must  ensure  that  Data  Link 
maintains  its  value  as  an  aid  to  the  controller  and  not  become  a 
source  of  distraction.  The  broad  range  of  ratings  and  comments 
regarding  Menu  Text  indicated  that  if  the  message  selection  tasks 
can  be  simplified,  this  service  will  be  extremely  valuable  and 
highly  acceptable  to  its  users. 

Primary  comments  supporting  high  ratings  of  both  controller 
preference  and  operational  effectiveness  for  the  transfer  of 
communication,  terminal  information  and  initial  contact  services 
cited  the  time  saved  and  reduction  in  frequency  congestion  produced 
by  converting  these  repetitive  and/or  lengthy  communications  to 
Data  Link.  Reasons  cited  for  less  than  optimum  ratings  included 
the  need  for  further  testing  of  the  services  to  insure  their  broad 
applicability. 

3.3  FINAL  DEBRIEFING. 

The  structured  discussion  session  which  followed  the  second  day  of 
simulation  testing  was  focused  on  the  subject's  overall  predictions 
about  the  impact  of  implementing  Data  Link  in  the  terminal  ATC 
environment,  operational  Data  Link  issues,  and  needs  for  future 
Data  Link  Test  Bed  Research  to  refine  and  evaluate  terminal 
services. 

3.3.1  Data  Link  Impact. 

The  subjects  noted  that  implementing  Data  Link  within  the  current 
ATC  system  would  produce  at  least  three  positive  effects.  In 
agreement  with  their  emphasis  on  the  problems  of  time  limitations 
in  terminal  airspace  expressed  earlier  in  the  study  and  in  their 
comments  regarding  workload,  the  test  subjects  felt  that  Data  Link 
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would  provide  them  with  more  available  time  on  the  radio 
frequencies  to  communicate  important  information  to  pilots.  Beyond 
this  direct  effect  of  reduced  frequency  congestion,  it  also  was 
noted  that  time  pressures  would  be  reduced  by  the  ability  to  send 
lengthy  clearances  and  advisories  with  a  single  manual  action. 
Finally,  Data  Link's  digital  communication  characteristics  and 
persistent  visual  displays  were  predicted  to  have  a  positive  impact 
on  the  accuracy  of  air-ground  communications. 

3.3.2  Data  Link  Delays  and  Voice  Usage. 

As  a  part  of  the  debriefing,  the  subjects  were  asked  to  reflect  on 
their  Test  Bed  simulation  experiences  in  order  to  discuss  how  voice 
and  Data  Link  were  used  in  the  mixed  equipage  tests,  and  to  obtain 
their  reactions  to  Data  Link  transmission  delays. 

The  subjects  reported  that  they  used  voice  procedures  with  Data 
Link  equipped  aircraft  in  order  to  adapt  to  time-critical 
situations,  to  resolve  transmission  failures  in  which  the  margin 
of  available  communication  time  had  become  too  short,  and,  in  one 
case,  to  correct  an  erroneous  uplink.  In  subsequent  discussion, 
one  of  the  three  controllers  who  had  manned  the  final  approach 
sector  in  the  simulation  runs  noted  that  he  was  uncomfortable  using 
Data  Link  during  this  phase  of  a  flight  since,  in  an  operational 
situation,  pilots  may  be  reluctant  to  utilize  a  "heads  down"  visual 
display  for  communications.  However,  other  controllers  noted  that 
Data  Link  may  be  useful  for  providing  the  final  approach  clearance. 
The  group  consensus  that  emerged  from  this  discussion  was  that  the 
optimal  use  of  Data  Link  during  final  approach  was  an  issue  that 
would  have  to  be  addressed  in  a  full  scale  operational  evaluation 
in  which  actual  pilots  and  nigh  fidelity  flight  simulators  were 
included . 

The  subjects  reported  no  specific  problems  with  Data  Link 
transmission  delays  in  the  context  of  the  relatively  low  complexity 
simulation  scenario  used  in  the  present  study.  As  in  earlier 
simulations  with  en  route  controllers,  the  terminal  controllers 
felt  that  they  were  able  to  accommodate  delays  by  uplinking 
messages  in  a  more  anticipatory  fashion  and  by  switching  to  voice 
when  necessary.  However,  it  was  agreed  that  future  Test  Bed 
evaluations  would  have  to  be  conducted  to  determine  the  effect  of 
delays . 

3.3.3  Future  Test  Requirements. 

Beyond  an  agreement  that  the  service  design  modifications  which 
emerged  from  the  present  study  will  require  a  second  mini  study 
evaluation,  the  controllers  also  emphasized  a  need  to  develop  more 
complex  simulation  scenarios  for  future  testing.  Specific  features 
of  the  testing  environment  that  were  noted  as  requirements 
included:  (a)  scenarios  as  long  as  90  minutes  to  assess  realistic, 
long  term  effects  on  controller  performance,  (b)  increased  traffic 
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loads  and  tighter  spacing  requirements,  and  (c)  the  inclusion  of 
anomalies  and  errors  in  the  testing  protocols  to  determine  whether 
Data  Link  displays  of  information  such  as  the  initial  contact 
downlink  have  sufficient  alerting  value  to  the  controller  under 
operational  conditions. 

4.  CONCLUSIONS. 

The  results  of  the  study  presented  in  this  report  warrant  a  number 
of  specific  and  general  conclusions  regarding  the  initial  terminal 
ATC  services  and  requirements  for  future  research  on  this  group  of 
Data  Link  applications. 

a.  Based  on  their  simulation  experiences  in  the  ARTS  IIIA  Data 
Link  Test  Bed,  the  controllers  who  participated  in  this  study 
concluded  that  Data  Link  would  have  strong,  positive  effects  on 
terminal  operations  if  implemented  in  the  current  ATC  system. 
These  effects  would  include  a  significant  reduction  in  voice  radio 
frequency  congestion,  simplified  transfer  of  lengthy  advisory 
messages  and  clearances,  and  a  reduction  in  communication  errors. 

b.  The  subjects  estimated  that  terminal  Data  Link  services 
would  reduce  controller  workload.  Projections  of  average 
controller  workload  in  a  60  percent  Data  Link  aircraft  equipage 
environment  were  significantly  lower  than  comparative  estimates  of 
controller  workload  in  the  current,  exclusively  voice  radio, 
communications  environment. 

c.  The  Initial  Contact,  Transfer  of  Communication,  Terminal 
Information,  and  Menu  Text  services,  as  modified  by  controller 
inputs  during  this  study,  received  high  controller  ratings  on  the 
dimensions  of  operational  effectiveness  and  acceptability  to  field 
controllers.  None  of  the  controllers  rated  any  of  the  service 
designs  as  operationally  unsuitable  or  completely  unacceptable  to 
users.  In  addition,  the  median  ratings  for  all  services  were 
either  the  highest  or  second  highest  possible  scores  on  the  7- 
point  operational  effectiveness  and  controller  preference  scales. 

d.  Several  modifications  and  enhancements  to  the  Data  Link 
service  designs  were  agreed  upon  during  the  study.  In  general, 
these  alterations  were  suggested  by  the  subjects  as  necessary  for 
future  evaluation,  and  not  as  final  design  specifications.  The 
most  important  of  the  desired  modifications  included  the  following: 

1.  Both  a  Data  Link  transaction  status  list  and  a 
transaction  history  list  should  be  added  to  the  ARTS  IIIA 
implementation  as  potential  supplements  to  the  tested  data  block 
transaction  displays.  Furthermore,  all  transaction  displays  should 
be  expanded  to  provide  more  detailed,  and  in  some  cases  more 
persistent,  information  regarding  the  outcome  of  each  transaction. 
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2.  The  lack  of  a  consensus  on  appropriate  symbology  for 
the  Data  Link  equipage  and  eligibility  display  will  necessitate 
the  maintenance  of  a  capability  to  manipulate  available  design 
options  in  future  studies. 

3.  The  Terminal  Information  service  should  be  modified  to 
permit  the  combined  uplink  of  any  available  message  on  the  list 
with  Menu  Text  selections.  In  addition,  the  ability  to  reassign 
any  of  the  messages  in  the  list  to  the  default  status  should  be 
added. 


4.  The  Transfer  of  Communication  service  should  be 
modified  to  permit  Data  Link  communications  during  the  period 
between  the  acceptance  of  a  sector  hand  off  and  the  transferring 
controller's  input  to  uplink  the  new  radio  frequency. 

5.  The  menu  by-pass  feature  introduced  during  the  study 
should  be  retained  and  enhanced  by  permitting  combined  clearances 
in  a  single  uplink.  In  addition,  fixed  menu  items  were  requested 
which  contain  predetermined  combined  speed,  heading,  and  altitude 
instructions . 

e.  Beyond  the  design  enhancement  recommendations  and  expert 
projections  that  were  derived  from  this  study,  the  results  also 
revealed  some  open  design  issues  which  may  have  a  large  impact  on 
the  performance  capabilities  of  the  terminal  Data  Link  controller. 
It  is  concluded  that  the  following  issues  will  require  specific 
research  attention  in  future  design  and  operational  evaluation 
research: 

1.  Menu  Usage  and  Design.  The  results  of  the  study 
indicate  that  individual  differences  may  exist  among  controllers 
in  their  preferred  approach  to  uplinking  ATC  clearance  messages. 
Some  of  the  test  controllers  preferred  to  develop  a  predefined  menu 
of  messages  which  would  cover  all  clearances  that  they  wished  to 
issue.  Others  felt  that  the  menu  should  be  limited  to  only  the 
messages  which  were  used  frequently  and  repetitively,  and  that  the 
by-pass  method  be  employed  as  needed  for  composing  other  speed, 
heading,  and  altitude  clearances.  If  the  option  to  permit  lengthy 
menus  is  exercised,  controller  comments  recorded  during  this  study 
indicate  that  care  must  be  used  during  continued  design  efforts  to 
avoid  the  potentially  excessive  memory  and  visual  search  demands 
which  can  accompany  the  use  of  these  data  entry  tools.  Possible 
solutions  may  include  the  use  of  mnemonic  devices  for  assigning 
messages  to  item  designators  and  the  spatial  separation  of  menu 
items  to  facilitate  the  task  of  locating  a  required  message. 

2.  Information  Display.  Data  Link  service  design  efforts 
in  both  en  route  and  terminal  environments  have  shown  that 
information  regarding  the  status  of  Data  Link  transactions  should 
be  presented  in  a  display  location  which  does  not  require  diversion 
of  attention  from  the  rac’ar  targets.  This  finding  led  to  the  use 


32 


of  the  data  block  as  a  primary  Data  Link  display  area.  However, 
controllers  have  tempered  this  requirement  with  a  warning  that  the 
addition  of  information  to  the  data  block  can  carry  a  penalty  of 
increased  visual  clutter,  and  present  a  source  of  distraction  to 
the  viewer. 

These  apparently  opposing  requirements  have  resulted  in 
data  block  displays  for  Data  Link  transactions  which  are  limited 
to  the  minimum  information  possible,  and  which  tend  to  present  data 
only  when  an  exception  to  the  normal  sequence  of  events  occurs. 
The  results  of  the  present  study  indicated  that  this  design 
philosophy  may  require  modification  to  insure  that  sufficient 
transaction  information  is  available  to  the  controller  (e.g.,  the 
display  of  the  cause  of  a  transaction  failure) .  Because  of  this, 
continued  research  attention  will  be  needed  to  provide  Data  Link 
displays  which  provide  all  necessary  information  to  support 
controller  decisions  while  avoiding  the  disruption  of  other 
monitoring  functions. 

3.  Short  Term  Memory.  Effective  monitoring  and  control 
of  an  aircraft's  progress  place  a  demand  on  the  controller  to 
remember  the  content  of  prior  clearances.  While  this  requirement 
applies  equally  to  voice  and  Data  Link  communications,  the  suojects 
in  the  present  study  often  emphasized  the  need  for  displays  which 
will  confirm  that  a  message  they  had  intended  to  send  had,  in  fact, 
been  sent  and  that  the  pilot  had  responded  affirmatively.  Examples 
of  desired  design  features  which  reflect  this  requirement  include 
the  history  list  and  a  persistent  display  of  the  WILCO  response  in 
the  data  block. 

At  present,  it  is  not  clear  whether  these  aids  may  be 
needed  because  of  differential  short  term  memory  for  spoken  and 
manually  entered  messages,  the  visual  nature  of  Data  Link 
communications  displays,  or  lack  of  long  term  experience  with  Data 
Link  inputs.  Since  memory  failures  in  the  present  voice-only 
communications  channel  are  known  to  be  common,  it  is  also  possible 
that  the  digital  nature  of  Data  Link  communications  simply  makes 
it  possible  to  provide  a  more  generally  needed  memory  aid  to 
controllers  which  is  not  available  in  the  current  system. 
Regardless  of  the  reason,  the  data  collected  thus  far  support  the 
conclusion  that  continued  attention  be  given  in  design  evaluation 
studies  to  the  requirement  for  controller  information  displays 
which  can  prevent  short  term  memory  lapses. 

5.  RECOMMENDATIONS. 

The  following  recommendations  for  future  efforts  under  the  FAA 
terminal  Data  Link  program  are  derived  from  the  findings  and 
conclusions  of  the  present  research: 

a.  The  results  of  this  study  indicated  that  the  initial 
terminal  ATC  Data  Link  services  have  the  potential  to  reduce 
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frequency  congestion  and  controller  workload  and  are  compatible 
with  the  existing  NAS  terminal  controller  workstation  and  computer. 
Therefore,  it  is  recommended  that  continued  development  and 
operational  evaluation  research  be  pursued  to  implement  these 
services  at  the  earliest  possible  date. 

b.  It  is  also  recommended  that  work  be  initiated  to  verify 
the  technical  feasibility  of  implementing  the  Data  Link  services 
in  the  operational  ATC  system.  These  efforts  should  include  the 
empirical  tests  on  ARTS  IIIA  capabilities  outlined  in  appendix  A, 
as  well  as  analyses  of  the  impact  on  future  terminal  systems. 

c.  The  service  design  modifications  and  enhancements 
identified  during  the  present  study  should  be  implemented  in  the 
ARTS  IIIA  Test  Bed  for  review  and  evaluation  by  the  ATDLVT  in  a 
second  mini  study. 

d.  Simulation  scenarios  and  testing  protocols  should  be 
developed  for  operational  evaluation  purposes.  These  testing 
conditions  should  be  designed  to  test  the  robustness  of  the  Data 
Link  service  designs  under  high  air  traffic  loads,  minimum  aircraft 
spacing  distances,  long  duration  controller  work  periods,  and 
anomalous  error  conditions. 

e.  Future  research  should  continue  to  focus  on  the  human 
factors  issues  which  may  impact  the  effectiveness  of  terminal  Data 
Link  services.  These  include  the  control  of  visual  search  demands 
in  the  use  of  menu  functions,  insurance  of  optimum  information 
content  in  transaction  status  displays,  and  minimization  of  short 
term  memory  requirements. 

f.  Future  design  and  testing  efforts  should  be  devoted  to 
resolving  issues  that  are  common  betw.een  the  terminal  and  en  route 
Data  Link  designs.  Future  testing  should  include  an  integrated 
terminal  and  en  route  test  effort  to  verify  coherent  system  design 
between  terminal  and  en  route  facilities. 
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APPENDIX  A 

ANALYSIS  OF  DATA  LINK  IMPACT  ON  THE  ARTS  IIIA 


EVALUATION  IMPACT 


The  evaluation  of  a  new  function  such  as  Terminal  Data  Link  v/c”ld 
not  Le  complete  witnout  considering  the  impact  it  would  have  on  the 
ARTS  IIIA  operational  system.  There  are  three  considerations: 
available  memory,  available  processing  time,  and  display  refresh. 

MEMORY . 

The  ARTS  IIIA  system  can  be  configured  for  a  single  sensor  or  a 
dual  sensor  system.  The  ARTS  IIIA  was  originally  equipped  with 
128k  of  core  memory  for  a  single  sensor  system  and  160k  of  core 
memory  for  a  dual  sensor  system,  and  was  later  increased  to  160k 
and  192k,  respectively.  More  recently  as  a  result  of  the  Interim 
Support  Plan  (ISP) ,  currently  under  contract  and  in  progress,  the 
ARTS  IIIA  is  being  upgraded  to  256k  of  semiconductor  memory  at  all 
sites.  Busy  sites  such  as  Chicago  O' Hare  and  Dallas/Fort  Worth  are 
already  operating  with  the  new  memory. 

The  ARTS  IIIA  system  currently  operating  in  the  field,  A3. 04, 
requires  128k  of  memory  for  a  single  sensor  and  176k  for  a  dual 
sensor.  This  allows  128k  for  expansion  in  a  single  sensor  system 
an  8  0  in  a  dual  sensor  system.  The  current  Data  Link  services 
would  require  no  more  than  10k  of  the  memory  that  is  available. 

PROCESSING  TIME. 

ARTS  IIIA  processing  capacity  caused  a  great  deal  of  concern  in 
recent  years.  However,  as  a  result  of  several  software  changes  to 
make  the  software  more  efficient  and  the  implementation  of  the  high 
speed  semiconductor  memory  mentioned  above,  the  concern  of 
processing  capacity  has  been  put  to  rest.  For  example,  Chicago 
O' Hare,  one  of  the  busiest  airports  in  the  country  was  at  capacity 
prior  to  the  software  and  memory  improvements.  That  system  is 
operating  at  approximately  65  percent  during  the  busiest  periods 
of  traffic.  The  Data  Link  functions  currently  under  test  are 
predominately  keyboard  triggered.  Keyboard  entries  have  a  low 
impact  on  processing  time.  It  is  the  events  triggered,  such  as 
uplink  messages,  that  may  cause  somewhat  of  an  impact.  However, 
the  impact  is  not  expected  to  be  more  than  5  percent  maximum.  This 
is  a  best  estimate.  It  would  be  necessary  to  run  two  simulations 
with  identical  traffic  loads,  one  with  Data  Link  and  one  without, 
utilizing  the  system  performance  monitor  to  determine  the  actual 
difference  in  processing  time. 

DISPLAY  REFRESH. 

The  ARTS  IIIA  Display  Entry  Devices  (DEDs)  display  are  1960 's 
technology  and  were  not  designed  to  handle  the  data  rates  of  the 
busier  facilities  in  the  1990 's.  Many  of  the  busier  sites  have 
complained  of  display  flicker.  Fortunately,  the  Interim  Support 
Plan  contract,  mentioned  earlier,  includes  the  purchase  of  150  Full 
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Digital  Arts  Displays  (FDADs)  to  be  installed  at  the  busier 
facilities.  As  a  result,  the  additional  tabular  lists  of  TI  and 
MT  messages  and  the  occasional  third  line  of  the  data  block  will 
not  impact  display  refresh  at  sites  using  FDAD  displays.  In  any 
case,  should  some  medium  traffic  facility  have  a  flicker  problem, 
it  could  be  resolved  with  the  purchase  of  additional  FDAD 1 s . 

TERMINAL  DATA  LINK  FACT  SHEET 

The  current  services  available  with  the  A3. 03  software 
modifications  are  Initial  Contact,  Terminal  Information,  Message 
Text,  and  Transfer  Control.  The  following  is  a  list  of  ARTS  IIIA 
additional  memory  requirements  for  the  current  services: 

Program  Storage 
Keyboard 
Display  Output 
Data  Link  Input 
Data  Link  Output 
Misc.  Patches 

Data  Storage 


Data  Link  Inter¬ 
facility  Data 

TI/MT  Messages  (4  TI, 

12  MT) ,  86  Words  Per 
Controller 
10  Controllers 

Central  Track  Store 
5  Additional  Words 
Per  Track,  300  Tracks 

Display  Output  Buffers 
63  Words  Per  Controller 

10  Controllers  630  20  Controllers  1260 

The  ARTS  IIIA  System  currently  in  the  field  (A3. 04)  requires 
approximately  128k  of  memory  for  a  single  sensor  system  and  176k 
of  memory  for  a  dual  sensor  system.  The  Interim  Support  Plan  (ISP) 
contract  with  UNISYS  provides  for  all  ARTS  IIIA  systems  to  be 
upgraded  to  256k  of  high  speed  semiconductor  memory.  This  upgrade 
will  easily  allow  sufficient  space  for  all  the  Data  Link  functions 
to  be  added  without  interfacing  with  other  enhancements. 


2354  words  total 
758 
612 
263 
495 
226 


Single  Sensor 
3490  Words 


Dual  Sensor 
6680  Words 


500 


700 


860 


20  Controllers  1720 


1500 


600  Tracks 


3000 
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APPENDIX  B 


DESCRIPTIONS  OF  THE  TESTED  DATA  LINK  SERVICES 


DESCRIPTION 


Operational  descriptions  of  each  of  the  Data  Link  services 
evaluated  in  this  study  are  presented  ’ n  the  following  sections  of 
this  appendix'.  These  descriptions  do  not  reflect  enhancements  and 
modifications  to  the  designs  which  were  recommended  by  the  subject 
controllers  during  the  design  review. 

The  photographs  which  follow  the  descriptions  present  an  overall 
view  of  the  ARTS  IIIA  workstation  in  the  Data  Link  Test  Bed  with 
Data  Link  displays  shown  on  the  radar  screen,  a  closeup  of  the  data 
block  with  eligibility  and  equipage  symbology  on  the  first  line  and 
transaction  information  presented  on  the  third  line,  and  a  close 
up  of  the  Terminal  Information  Service  and  Menu  Text  lists. 

Note  in  the  following  service  descriptions: 

1.  The  SLEW  command  should  be  interpreted  as  the  action  sequence 
of  acquiring  the  target  with  the  trackball  and  pressing  the 
trackball  enter  key. 

2.  The  ENTER  command  should  be  interpreted  as  pressing  the 
keyboard  ENTER  key. 

3.  Data,  as  shown  in  a  display,  are  presented  in  quotation  marks. 
The  quotation  marks  are  not  part  of  the  display. 

4.  For  all  services,  only  one  Data  Link  transaction  per  aircraft 
may  be  in  progress  at  a  time  —  the  controller  may  not  uplink  a  new 
message  until  the  previous  service  has  been  WILCOed  or  a  FAIL  has 
been  cleared  from  the  data  block  display. 

INITIAL  CONTACT  (IC) 


SERVICE  DESCRIPTION. 

1.  Initiation  of  IC. 

The  request  for  assigned  altitude  is  automatically  uplinked  to  an 
aircraft  when  the  pilot  downlinks  a  WILCO  message  to  a  preceding 
transfer  of  communication. 

2.  Display  on  Downlink  of  Assigned  Altitude. 

When  the  aircraft  downlinks  its  assigned  altitude,  the  third  line 
of  the  data  block  displays  the  character  IC  followed  by  the 
altitude  value  (e.g.,  IC  110).  The  altitude  value  flashes  to 
capture  the  controller's  attention  and  signal  a  required  response. 

This  response  may  be  a  Terminal  Information  (TI)  message, 
initiation  of  a  transfer  of  communication,  one  or  more  Menu  Text 
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(MT)  messages,  or  a  combination  of  a  TI  message  and  MT  messages. 
Initiation  of  the  uplink  immediately  deletes  the  IC  display  in  the 
third  line  of  the  data  block. 

3.  Display  on  Failure  to  Downlink  Assigned  Altitude. 

If  the  aircraft  system  fails  to  technically  acknowledge  the 
altitude  request  or  fails  to  downlink  the  assigned  altitude  within 
40  seconds  of  receiving  the  uplinked  request,  the  third  line  of  the 
data  block  displays  the  characters  IC  FAIL.  FAIL  flashes  to  signal 
a  required  controller  response. 

4.  Inputs  to  Resend  Altitude  Request. 

If  a  response  failure  occurs,  the  controller  may  resend  the 
altitude  request  by  a  SLEW  action. 

5.  Inputs  to  Clear  IC  FAIL  Display. 

The  IC  FAIL  display  may  be  manually  deleted  by  pressing  the  D/L  key 
(F9) ,  iollowea  by  a  SLEW  action. 

TERMINAL  INFORMATION  SERVICE  (TI) 

SERVICE  DESCRIPTION. 

1.  Initiation  of  TI . 

A  TI  message  can  be  sent  at  any  time  through  a  controller  input 
action.  When  used  immediately  after  receiving  an  initial  contact 
message  from  an  aircraft  (IC  and  an  altitude  value  displayed  in 
third  line  of  data  block) ,  simplified  SLEW  inputs  can  be  used  to 
send  commonly-used  TI  messages. 

2.  TI  List  Display. 

TI  messages  are  selected  from  a  list  display  containing  several 
optional  items.  Each  message  (up  to  29  characters)  is  presenteu  on 
a  single  line  of  the  list  preceded  by  a  identifying  digit  or  the 
letter  D.  The  message  identified  by  D  is  a  commonly-used  default 
message.  The  remaining  items  are  numbered  consecutively  beginning 
with  1. 

3.  Inputs  to  Reposition  TI  List. 

The  TI  list  can  be  moved  to  any  position  on  the  ARTS  display  by 
pressing  the  MULTIFUNCTION  key  (F7) ,  typing  TC  and  SLEW  to  position 
the  list. 
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4.  Inputs  to  Suppress/Retrieve  TI  List. 

The  TI  lisr  can  be  removed  from  the  ARTS  display  by  pressing  the 
D/L  key  (F9)  ,  typing  T  and  ENTER.  Repeating  this  sequence  will 
retrieve  the  list. 

5.  Inputs  to  Send  TI  -  Default  Message. 

When  a  successful  initial  contact  has  been  completed  and  the  third 
line  of  the  data  block  displays  IC  followed  by  an  altitude  value, 
the  default  TI  message  may  be  uplinked  by  a  SLEW  action. 

6.  Inputs  to  Send  TI  -  Non-Default  Message. 

Non-default  messages  can  be  uplinked  by  typing  T,  the  message 
number  from  the  TI  list  (e.g.,  3)  and  SLEW. 

7.  Displays  on  TI  Uplink. 

If  a  TI  message  is  sent  when  IC  and  an  altitude  value  are  shown  in 
the  third  line  of  the  data  block,  the  entry  deletes  the  third  line 
and  replaces  it  with  TI  followed  by  the  message  number  selected 
from  the  list  for  uplink. 

S.  Displays  After  Pilot  WILCO. 

Upon  receipt  of  a  downlinked  WILCO  response  from  the  aircraft,  the 
TI  message  number  display  in  the  third  line  of  the  data  block  is 
deleted . 

9.  Displays  on  Failed  Response. 

If  the  aircraft  does  not  technically  acknowledge  the  uplink  or  the 
pilot  fails  to  respond  within  40  seconds  after  receiving  the  TI 
message,  TI  FAIL  appears  in  the  third  line  of  the  data  block  with 
the  FAIL  characters  flashing. 

10.  Inputs  to  Resend  TI. 

The  controller  may  resend  a  TI  message  after  a  response  failure  by 
a  SLEW  action. 

11.  Inputs  to  Clear  TI  FAIL  Display. 

The  TI  FAIL  display  can  be  deleted  by  pressing  the  D/L  (F9)  key  and 
SLEW. 

12.  Inputs  to  Change  Default  Message. 

The  default  message  can  be  changed  only  by  typing  a  new  message  on 
the  default  message  line. 
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13.  Combining  TI  with  MT  Message(s). 

The  default  TI  message  can  be  sent  together  with  one  or  more  MT 
messages  by  typing  T  prior  to  the  Menu  Text  entry  (see  MT 
description) . 

TRANSFER  OF  COMMUNICATIONS  (TOC) 

SERVICE  DESCRIPTION. 

1.  Initiation  of  TOC. 

The  TOC  message  containing  a  new  radio  frequency  for  an  aircraft 
is  automatically  prepared  when  the  receiving  controller  accepts  a 
sector  hand  off. 

2.  Inputs  to  Send  TOC  -  Automatic. 

The  TOC  is  automatically  uplinked  upon  acceptance  of  the  handoff 
if  the  sending  controller  presses  the  HANDOFF  key  (F5) ,  types  the 
receiving  sector's  Controller  Symbol,  and  SLEW.  Any  other  currently 
acceptable  ARTS  input  sequence  normally  used  to  initiate  a  handoff 
will  result  in  the  same  automatic  uplink  of  TOC  (e.g. ,  F5-ACID- 
Controller  Symbol-  ENTER) . 

3.  Display  on  Automatic  TOC  Uplink. 

TC  is  displayed  in  the  third  line  of  the  data  block  when  the  TOC 
message  is  uplinked. 

4.  Inputs  to  Send  TOC  -  Manual. 

The  controller  may  initiate  the  sector  handoff  but  reserve 
communications  eligibility  until  a  later  time  by  using  a  manual 
procedure.  Pressing  the  HANDOFF  key  (F5) ,  typing  the  receiving 
sector's  Controller  Symbol,  the  letter  H  and  SLEW,  initiates  the 
handoff,  but  holds  the  TOC  message  until  the  controller  takes  a 
SLEW  action. 

5.  Display  on  Manual  TOC. 

TC  H  is  displayed  in  the  third  line  of  the  data  block  when  the 
manual  TOC  entry  is  made.  When  the  controllers  slews  the  aircraft, 
the  H  is  deleted  and  the  message  is  uplinked. 

6.  Display  After  Pilot  WILCO. 

In  both  automatic  and  manual  procedures,  the  TC  display  in  the 
third  line  of  the  data  block  is  deleted  when  the  aircraft  downlinks 
a  WILCO  response. 


8.  Display  on  Failed  Response. 

If  the  aircraft  fails  to  technically  acknowledge  the  TOC  or  if  the 
pilot  fails  to  WILCO  the  message  within  40  seconds  of  receipt,  TC 
FAIL  appears  in  the  third  line  of  the  data  block  with  the  FAIL 
characters  flashing. 

9.  Inputs  to  Resend  TOC. 

The  controller  may  resend  a  failed  TOC  message  by  a  SLEW  action. 

10.  Inputs  to  Clear  TOC  FAIL  Display. 

The  controller  may  delete  the  TC  FAIL  display  by  pressing  the  D/L 
key  (F9)  and  a  SLEW  action. 

11.  Inputs  to  Acquire  Data  Link  Eligibility. 

The  controller  may  acquire  (steal)  eligibility  for  Data  Link 
communications  by  pressing  the  D/L  key  (F9) ,  typing  the  letters  OK 
and  a  SLEW  action. 


MENU  TEXT  (MT) 


SERVICE  DESCRIPTION. 

1.  Initiation  of  MT. 

The  MT  service  permits  the  controller  to  uplink  one  or  more 
clearances  contained  in  a  menu  list  by  manually  selecting  the 
required  messages. 

2.  MT  List  Display. 

Available  MT  clearances  are  displayed  in  a  list  containing  several 
items.  Each  menu  item  (up  to  29  characters)  is  displayed  on  a 
single  line  preceded  by  an  identifying  number.  Messages  are 
selected  using  the  identifying  numbers. 

3.  Inputs  to  Reposition  MT  List. 

The  position  of  the  MT  list  on  the  ARTS  display  can  be  altered  by 
pressing  the  MULTIFUNCTION  key  (F7),  typing  TC,  and  completing  a 
SLEW  action  to  move  the  list. 

4.  Inputs  to  Suppress/Retrieve  MT  List. 

The  MT  list  can  be  removed  from  the  display  by  pressing  the  D/L  key 
( F9 )  and  typing  M  and  ENTER.  The  list  is  retrieved  using  the  same 
sequence  of  key  strokes. 
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5.  Inputs  to  Send  a  MT  Message. 

A  single  MT  item  can  be  uplinked  by  typing  M  followed  by  the  menu 
item  identifying  number,  and  a  SLEW  action. 

6.  Inputs  to  Send  Multiple  MT  Messages. 

Up  to  three  MT  menu  items  can  be  sent  in  a  single  uplink  by 
inserting  spaces  between  the  item  numhe'*s  (e.g.,  Ml  2  3  SLEW  would 
send  menu  items  1,  2,  and  3). 

7.  Inputs  to  Send  MT  Message (s)  Combined  with  TI. 

The  default  TI  message  can  be  sent  in  combination  with  up  to  two 
MT  items  by  adding  T  as  the  first  character  of  the  MT  input,  (e.g., 
TM1  2  SLEW  would  send  the  default  TI  message  and  menu  items  1  and 
2)  . 

8.  Displays  on  MT  Uplink. 

When  an  MT  uplink  is  initiated,  MT  appears  in  the  third  line  of  the 
data  block  followed  by  the  identifying  numbers  of  the  messages 
sent.  T  MT  followed  by  the  MT  messages  sent  is  displayed  when  the 
default  TI  message  is  combined  with  MT  messages. 

9.  Displays  After  Pilot  WILCO. 

A  downlinked  WILCO  to  an  MT  message  or  group  of  messages  deletes 
the  data  in  the  third  line  of  the  data  block. 

10.  Displays  on  Failed  Response. 

If  the  aircraft  fails  to  technically  acknowledge  an  MT  uplink  or 
the  pilot  fails  to  downlink  a  WILCO  response  within  40  seconds  of 
its  receipt,  MT  FAIL  appears  in  the  third  line  of  the  data  block 
with  the  FAIL  characters  flashing. 

11.  Inputs  to  Resend  MT. 

Upon  a  failed  response  to  a  MT  uplink,  the  controller  may  resend 
the  message  with  a  SLEW  action. 

12.  Inputs  to  Clear  MT  FAIL  Display. 

The  MT  FAIL  display  can  be  manually  deleted  by  pressing  the  D/L  key 
(F9)  and  making  a  SLEW  action. 

13.  Modifying  Numeric  Values  in  Menu  Items. 
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The  numeric  value  of  a  heading,  speed,  or  altitude  clearance  can 
be  changed  by  pressing  the  D/L  key  (F9) ,  typing  M  and  the 
one-digit  identifying  number  of  the  menu  item  to  be  changed,  typing 
the  new  numeric  value  (three  digits) ,  and  pressing  the  ENTER  key. 

If  a  SLEW  action  is  substituted  for  the  keyboard  ENTER,  the  menu 
item  will  be  changed  AND  uplinked  to  the  slewed  aircraft. 

14.  Bypassing  the  Menu. 

A  heading  (H) ,  altitude  (A)  or  speed  (S)  not  contained  in  the  menu 
can  be  uplinked  by  pressing  the  D/L  key,  typing  H,  A  or  S  followed 
by  the  three  digit  numeric  value  of  the  clearance  and  SLEW.  The 
third  line  of  the  data  block  displays  MT,  the  clearance  letter, 
and  the  three  digit  numerical  value  entered  (e.g.,  MT  A  110)  until 
the  message  is  WILCOed  or  FAILS. 
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FIGURE  B-2.  THE  DATA  BLOCK  TRANSACTION  AND  EQUIPAGE/ELIGIBILITY  DISPLAY 
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APPENDIX  C 


PROJECTIVE  RATING  MATERIALS 


INTRODUCTION  TO  THE  RATING  SCALES 


During  the  second  part  of  this  mini  study  you  will  be  controlling 
air  traffic  in  a  dynamic  scenario  using  current  voice  radio 
procedures  in  one  test  run  and  the  Data  Link  services  in  a  second 
test  run.  The  purpose  of  this  exercise  is  to  permit  you  to  make 
some  quantitative  judgments  about  the  Data  Link  services  and  their 
potential  effects  on  the  controller's  task. 

Because  most  of  you  have  directly  participated  in  the  development 
and  specification  of  the  terminal  Data  Link  services,  it  will  be 
extremely  important  for  you  to  make  every  effort  to  be  as  impartial 
as  possible  in  making  the  ratings  described  below.  In  your 
capacity  as  subjects  for  this  study,  you  are  representing  many 
other  terminal  controllers  whose  work  will  be  affected  by  the 
introduction  of  Data  Link.  Do  your  best  to  base  your  ratings  purely 
on  your  professional  experience  as  controllers  and  on  the  exposure 
that  you  will  be  given  to  the  initial  Data  Link  services  in  the 
test  bed. 

Your  judgments  will  be  made  using  a  set  of  three  rating  scales 
designed  to  measure  three  different  factors;  (1)  the  Operational 
Effectiveness  and  Suitability  of  each  service,  (2)  the 
Acceptability  and  Controller  Preference  for  each  service  design, 
and  (3)  the  effect  of  the  combined  services  on  controller  Workload. 
Each  of  these  factors  is  described  below: 

OPERATIONAL  EFFECTIVENESS  AND  SUITABILITY. 

The  first  scale  (figure  C-l)  asks  for  a  rating  of  the  effectiveness 
and  suitability  of  each  of  the  four  Data  Link  services  designs. 
To  evaluate  this  factor  you  should  assess  the  degree  to  which  the 
service  implementation  that  you  have  worked  with  in  the  Test  Bed 
could  effectively  accomplish  the  task  it  was  designed  to  do.  Thus, 
you  should  consider  whether  the  design  provides  all  of  the 
functions  and  capabilities  that  would  be  needed  to  meet  operational 
requirements  in  a  field  setting. 

To  complete  the  scale,  first  decide  whether  or  not  the  design  is 
capable  of  meeting  any  of  its  operational  requirements  of  the 
service.  If  it  is  not  at  all  operationally  suitable,  place  an  X 
in  the  box.  If  the  design  can  meet  some,  or  all,  operational 
requirements,  place  an  X  on  the  line  below  the  number  that  best 
describes  its  effectiveness.  Your  rating  can  range  from  1  (highly 
effective)  to  7  (meets  minimal  requirements) . 

CONTROLLER  ACCEPTANCE  AND  PREFERENCE. 

The  second  scale  (figure  C-l)  asks  for  a  rating  of  the 
acceptability  of  a  service  design  to  controllers  and  of  how 
preferable  it  is.  To  evaluate  this  factor  you  should  consider  the 
extent  to  which  the  displays,  inputs,  and  procedures  for  the 


DATA  LINK  SERVICE 


Use  the  scales  below  to  rate  the  terminal  service  indicated 
above.  On  the  first  scale,  check  the  box  if  this  service 
is  unsuitable  for  operational  use.  If  the  service  can  meet 
some,  or  all  operational  requirements,  check  the  space  below 
the  number  that  best  describes  its  effectiveness. 

OPERATIONAL  EFFECTIVENESS  /  SUITABILITY 


Highly 

Effective 


Meets  Most 
Operational 
Requirements 


Minimally 

Effective 


♦ 


t 

Not 

Operationally 

Suitable 


On  this  scale,  regardless  of  the  service’s  effectiveness, 
check  the  box  if  the  way  in  which  the  service  is  designed 
would  be  completely  unacceptable  to  controllers.  If  the 
service  design  is  acceptable,  check  the  space  below  the 
number  that  best  describes  your  preference  for  the  design. 


CONTROLLER  ACCEPTANCE  /  PREFERENCE 


Highly 

Preferred 

♦ 


Moderately 

Preferred 

♦ 


Acceptable, 

But  Not 
Preferred 

♦ 

t 

Completely 

sm=mmB  Unacceptable 


Please  use  the  following  page  to  comment  on  your  ratings. 

FIGURE  C-I.  EFFECTIVENESS /PREFERENCE  RATING  FORM 
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service  would  be  usable  by  field  controllers  and  adaptable  to  their 
styles  and  methods  of  controlling  air  traffic. 

To  complete  this  scale,  first  decide  whether  the  design  would  be 
acceptable  or  unacceptable  to  controllers.  If  it  is  completely 
unacceptable,  place  an  X  in  the  box.  If  it  is  acceptable,  rate 
the  design  by  placing  an  X  on  the  line  below  the  number  which  best 
describes  your  preference.  Your  rating  can  range  from  1  (highly 
preferred)  to  7  (acceptable,  but  not  preferred) . 

Note:  When  making  your  ratings,  remember  to  make  independent 
judgments  of  controller  preference  and  operational  suitability. 
Although  these  two  dimensions  may  tend  to  vary  together,  a  service 
design  could  provide  all  of  the  functions  needed  to  meet 
operational  requirements  and  still  be  poorly  suited  to  the 
controller's  way  of  doing  his  job.  Likewise,  a  design  could  be 
easy  to  use,  but  have  loopholes  which  prevent  it  from  covering  all 
operational  situations. 

WORKLOAD. 

The  final  rating  that  you  will  be  asked  to  make  is  a  projection 
about  the  effect  that  you  think  the  four  combined  Data  Link 
services  will  have  on  the  workload  of  the  terminal  air  traffic 
controller.  To  make  this  projection  you  will  be  asked  to  use  your 
personal  experience  estimate  the  average  workload  of  a  controller's 
job  on  a  typically  busy  day.  Based  on  your  experience  in  the  Test 
Bed,  you  will  then  be  asked  to  provide  another  estimate  of  the 
workload  that  would  be  experienced  under  the  same  conditions  if  the 
four  tested  Data  Link  services  were  available  to  the  controller. 

The  scale  that  you  will  use  to  make  these  estimates  is  known  as 
Projective  Subjective  Workload  Assessment  Technique  (PROSWAT) . 
PROSWAT  was  developed  as  a  method  for  collecting  quantified 
subjective  data  on  how  hard  a  person  feels  he  would  have  to  work 
when  using  different  procedures  and  equipment  to  perform  his 
duties . 

If  you  examine  the  PROSWAT  rating  scale,  you  will  notice  that 
PROSWAT  defines  workload  in  terms  of  a  combination  of  three 
different  dimensions  that  contribute  to  the  subjective  feeling  of 
"working  hard."  A  workload  rating  in  PROSWAT  is  accomplished  by 
selecting  a  1,  2,  or  3  on  EACH  of  the  three  scales  representing 
the  dimensions  of  TIME  LOAD,  MENTAL  EFFORT,  and  PSYCHOLOGICAL 
STRESS. 

Each  of  these  dimensions  and  their  levels  are  described  below: 
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TIME  LOAD. 


Time  Load  refers  to  the  fraction  of  the  total  time  that  you  are 
busy.  When  Time  Load  is  low,  sufficient  time  is  available  to 
complete  all  of  your  mental  work  with  some  time  to  spare.  As  Time 
Load  increases,  spare  time  drops  out  and  some  aspects  of 
performance  overlap  and  interrupt  one  another.  This  overlap  and 
interruption  can  come  from  performing  more  than  one  task  or  from 
different  aspects  of  performing  the  same  task.  At  high  levels  of 
Time  Load,  several  aspects  of  performance  often  occur 
simultaneously,  you  are  busy,  and  interruptions  are  very  frequent. 

Time  Load  is  rated  on  the  three  point  scale  below: 

\ 

1.  Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

( 

2.  Occasionally  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  frequently. 

3.  Almost  never  have  spare  time.  Interruptions  or  overlap  among 
activities  are  very  frequent,  or  occur  all  the  time. 

MENTAL  EFFORT  LOAD. 

As  described  above.  Time  Load  refers  to  the  amount  of  time  one  has 
available  to  perform  a  task  or  tasks.  In  contrast,  Mental  Effort 
Load  is  an  index  of  the  amount  of  attention  or  mental  effort 
required  by  a  task  regardless  of  the  number  of  tasks  to  be 
performed  or  any  time  limitations.  When  Mental  Effort  Load  is  low, 
the  concentration  and  attention  required  by  a  task  is  minimal  and 
performance  is  nearly  automatic.  As  the  demand  for  mental  effort 
increases  due  to  task  complexity  or  the  amount  of  information  which 
must  be  dealt  with  in  order  to  perform  adequately,  the  degree  of 
concentration  and  attention  required  increases.  High  Mental  Effort 
Load  demands  total  attention  or  concentration  due  to  task 
complexity  or  the  amount  of  information  that  must  be  processed. 

Mental  Effort  Load  is  rated  using  the  3-point  scale  below: 

1.  Very  little  conscious  mental  effort  or  concentration  required. 

Activity  is  almost  automatic,  requiring  little  or  no  attention.  , 

2.  Moderate  conscious  mental  effort  or  concentration  required. 

Complexity  of  activity  is  moderately  high  due  to  uncertainty,  » 

unpredictability,  or  unfamiliarity.  Considerable  attention 

required. 

3.  Extensive  mental  effort  and  concentration  are  necessary.  Very 
complex  activity  requiring  total  attention. 
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PSYCHOLOGICAL  STRESS  LOAD. 


Psychological  Stress  Load  refers  to  the  contribution  to  total 
workload  of  any  conditions  that  produce  anxiety,  frustration,  or 
confusion  while  performing  a  task  or  tasks.  At  low  levels  of 
stress,  one  feels  relatively  relaxed.  As  stress  increases, 
confusion,  anxiety,  or  frustration  increase  and  greater 
concentration  and  determination  are  required  to  maintain  control 
of  the  situation. 

Psychological  Stress  Load  may  be  rated  on  the  3-point  scale  below: 

1.  Little  confusion,  risk,  frustration,  or  anxiety  exists  and  can 
be  easily  accommodated. 

2.  Moderate  stress  due  to  confusion,  frustration,  or  anxiety 
noticeably  adds  to  workload.  Significant  compensation  is  required 
to  maintain  adequate  performance. 

3.  High  to  very  intense  stress  due  to  confusion,  frustration,  or 
anxiety.  High  to  extreme  determination  and  self-control  required. 

Each  of  the  three  dimensions  just  described  contribute  to  workload 
during  performance  of  a  task  or  group  of  tasks.  Note  that  although 
all  three  factors  may  be  correlated,  they  need  not  be.  For 
example,  one  can  have  many  tasks  to  perform  in  the  time  available 
(High  Time  Load)  but  the  tasks  may  require  little  concentration 
(Low  Mental  Effort  Load) .  Likewise,  one  can  be  anxious  and 
frustrated  (High  Stress  Load)  and  have  plenty  of  spare  time  between 
relatively  simple  tasks.  Since  the  three  dimensions  contributing 
to  workload  are  not  necessarily  correlated,  please  treat  each 
dimension  individually  and  give  independent  assessments  of  the  Time 
Load,  Mental  Effort  Load,  and  Psychological  Stress  Load  that  you 
estimate  for  the  terminal  controller's  job  with  and  without  Data 
Link. 

The  form  that  you  will  be  using  to  make  your  PROSWAT  ratings  during 
the  Data  Link  test  session  is  shown  in  figure  C-2 .  Note  that  the 
descriptions  for  each  level  of  time,  effort  and  stress  load  have 
been  removed  to  save  space.  Should  you  need  to  review  these 
descriptions  during  testing,  a  copy  of  the  full  scale  will  be 
available  at  all  times. 


PROSWAT  WORKLOAD  RATINGS 


Based  on  your  operational  experience,  estimate  the  average 
workload  of  a  terminal  controller  on  a  typically  busy  day 
in  the  current,  voice-only  system: 

CURRENT  SYSTEM 


1  2  3 

TIME  LOAD  _  _  _ 

MENTAL  EFFORT  _  _  _ 

STRESS  _ 


Based  on  your  experiences  with  the  four  Data  Link  services 
in  the  Test  Bed,  estimate  the  average  workload  of  a  terminal 
controller  on  a  typically  busy  day  under  the  following 
condition: 

DATA  LINK  SERVICES  -  60%  EQUIPAGE 

1  2  3 


TIME  LOAD 
MENTAL  EFFORT 
STRESS 


Please  use  the  space  below  to  comment  on  your  ratings. 

i 


FIGURE  C-2 .  PROSWAT  RATING  FORM 


C-6 


PROSWAT  SCALE  DEVELOPMENT  CARD  SORT 


Now  that  you  are  familiar  with  the  three  rating  scales  that  will 
be  used  during  the  Data  Link  Test  session,  there  is  one  last 
procedure  that  must  be  completed  before  testing  can  begin.  This 
procedure  is  a  card  sorting  task  that  will  allow  us  to  interpret 
your  PROSWAT  workload  ratings.  We  will  be  asking  you  to  do  this 
task  during  the  briefing  that  will  take  place  to  review  the  rating 
scales. 

One  of  the  most  important  features  of  PROSWAT  is  its  unique  scoring 
t  system.  The  developers  of  PROSWAT  recognized  that  different  people 

have  different  conceptions  of  how  the  time,  effort  and  stress 
dimensions  combine  to  produce  an  overall  impression  of  low  and  high 
t  workload.  Because  of  these  differences,  a  special  card  sorting 

procedure  is  used  in  PROSWAT  to  define  a  distinctive  workload  scale 
for  each  person.  This  individualized  scale  greatly  improves  our 
ability  to  accurately  interpret  the  actual  workload  ratings  that 
you  will  be  making  during  the  test  sessions. 

In  order  to  develop  your  individual  scale,  we  need  information  from 
you  regarding  the  amount  of  workload  that  you  feel  is  produced  by 
various  combinations  of  the  three  levels  on  the  time,  effort,  and 
stress  dimensions.  We  get  this  information  by  having  a  person  rank 
order  a  set  of  cards.  Each  card  contains  a  different  combination 
of  the  levels  of  time  load,  mental  effort  load,  and  stress  load. 
Since  there  are  three  dimensions,  and  each  dimension  has  three 
levels,  there  are  27  cards  in  the  deck  that  you  will  be  sorting. 
Your  job  will  be  to  sort  the  cards  so  that  they  are  ranked 
according  to  the  level  of  workload  represented  by  each  card.  Thus, 
the  first  card  in  the  deck  will  represent  the  lowest  workload  and 
the  last  card  will  represent  the  highest  workload. 

In  completing  your  card  sorts,  please  consider  the  workload  imposed 
on  a  person  by  the  combination  represented  in  each  card.  Arrange 
the  cards  from  the  lowest  workload  condition  through  the  highest 
condition.  You  may  use  any  strategy  you  choose  in  rank  ordering 
the  cards.  One  strategy  that  proves  useful  is  to  arrange  the  cards 
into  a  number  of  preliminary  stacks  representing  High,  Moderate, 

)  and  Low  workload.  Individual  cards  can  be  exchanged  between 

stacks,  if  necessary,  and  then  rank  ordered  within  stacks.  Stacks 
can  then  be  recombined  and  checked  to  be  sure  that  they  represent 
r  your  ranking  of  lowest  to  highest  workload.  However,  the  choice 

of  strategy  is  up  to  you  and  you  should  choose  the  one  that  works 
best  for  you. 

There  is  no  school  solution  to  this  problem.  There  is  no  correct 
order.  The  correct  order  is  what,  in  your  judgment  best  describes 
the  progression  of  workload  from  lowest  to  highest  for  a  general 
case  rather  than  any  specific  event.  That  judgment  differs  for 
each  of  us.  The  letters  you  see  on  the  back  of  the  cards  are  to 
allow  us  to  arrange  the  cards  in  a  previously  randomized  sequence 
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so  that  everyone  gets  the  same  order.  If  you  examine  your  deck  you 
will  see  the  order  on  the  back  runs  from  A  through  Z  and  then  ZZ. 

Please  remember: 

1.  The  card  sort  is  being  done  so  that  a  workload  scale  may  be 
developed  for  you.  This  scale  will  have  a  distinct  workload  value 
for  each  possible  combination  of  Time  Load,  Mental  Effort  Load,  and 
Psychological  Stress  Load.  The  following  example  demonstrates  the 
relationship  between  the  card  sort  and  the  resulting  workload 
scale : 

TIME 
1 


EFFORT 

1 


STRESS 

1 


WORKLOAD  SCALE 
0.0 


•  •  •  • 

3  3  3  -  100.0 

Note  that  oth°r  than  the  fact  that  a  1-1-1  will  always  represent 
the  lowest  workload  and  that  a  3-3-3  will  always  represent  the 
highest  workload,  the  remaining  cards  could  occur  in  a  number  of 
orders.  Your  order  will  depend  on  how  you  weight  the  importance 
of  Time,  Effort,  and  Stress  dimensions. 

2.  When  performing  the  card  sorts,  use  the  descriptors  printed  on 
the  cards.  Please  remember  not  to  sort  the  cards  based  on  one 
particular  task  (such  as  controlling  air  traffic) .  Sort  the  cards 
according  to  your  general  view  of  workload  and  how  important  you 
consider  the  dimensions  of  Time,  Mental  Effort,  and  Psychological 
Stress  Load  to  be. 

3.  A  PROSWAT  rating  for  any  situation  consists  of  one  number  from 
each  of  the  three  dimensions.  For  example,  a  possible  PROSWAT 
rating  is  1-2-2.  This  represents  a  1  for  Time  Load,  a  2  for  Mental 
Effort  Load,  and  a  2  for  Psychological  Stress  Load. 

4.  We  are  not  asking  for  your  preference  concerning  Time,  Mental 
Effort,  and  Psychological  Stress  Load.  Some  people  may  prefer  to 
be  busy  rather  than  idle  in  either  Time  Load,  Mental  Effort  Load, 
or  Psychological  Stress  Load  dimension.  We  are  not  concerned  with 
this  preference.  We  need  information  on  how  the  three  dimensions 
and  the  three  levels  of  each  one  will  affect  the  level  of  workload 
as  you  see  it.  You  may  prefer  a  2-2-2  situation  instead  of  a 
1-1-1  situation.  However,  you  should  still  realize  that  the 
1-1-1  situation  imposes  less  workload  on  you  and  leaves  a  greater 
reserve  capacity. 

The  sorting  will  probably  take  30  minutes  to  an  hour.  Please  feel 
free  to  ask  questions  at  any  time. 


JUS  GOVEHNMfNI  PRjNTfNG  Of  PlCf 
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